From cristian.esquivias at gmail.com Tue May 1 06:24:49 2018 From: cristian.esquivias at gmail.com (Cristian Esquivias) Date: Mon, 30 Apr 2018 23:24:49 -0700 Subject: OutOfMemoryError on my BF interpreter Message-ID: With the release of 1.0.0-RC1 I thought I'd come back and see how Graal is doing. Great job; Graal is really starting to look like the platform everyone should pay attention to. To get back into it, I built a Brainf*ck interpreter with Truffle (https://github.com/cesquivias/bf-graal). I modeled it after simplelanguage so it should be easy to build and run. While running the mandelbrot program, about halfway through the program, the VM starts throwing OutOfMemoryError exceptions (one pasted below). It created a zip file under dumps/ but it contains a .bgv file I don't what to do with and a log file that just contains the stacktrace. I ran jvisualvm and took a heap dump. There are ~200k long[] objects alone created by graal taking up ~50% of the memory. Digging through the references it seems to be created/retained in org.graalvm.compiler.lir.alloc.lsra.LinearScanLifetimeAnalysisPhase$$Lambda$48#1 [GC root - Java frame] I'd provide some file from jvisualvm but the save button is grayed out. I hope this is enough info. My interpreter is up on GitHub so feel free to test it out. I tested this both on the community & enterprise edition on Ubuntu 18.04 in VirtualBox. - Cristian Thread[TruffleCompilerThread-12,10,main]: Compilation of BFRepeatingNode at 222b298d failed: java.lang.OutOfMemoryError: Java heap space at org.graalvm.compiler.lir.util.IndexedValueMap.(IndexedValueMap.java:55) at org.graalvm.compiler.lir.dfa.RegStackValueSet.(RegStackValueSet.java:62) at org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:70) at org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:46) at org.graalvm.compiler.lir.dfa.LocationMarker.processBlock(LocationMarker.java:107) at org.graalvm.compiler.lir.dfa.LocationMarker.build(LocationMarker.java:81) at org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:51) at org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:47) at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) at org.graalvm.compiler.lir.phases.LIRPhaseSuite.run(LIRPhaseSuite.java:96) at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) at org.graalvm.compiler.core.GraalCompiler.emitLowLevel(GraalCompiler.java:367) at org.graalvm.compiler.core.GraalCompiler.emitLIR0(GraalCompiler.java:336) at org.graalvm.compiler.core.GraalCompiler.emitLIR(GraalCompiler.java:295) at org.graalvm.compiler.core.GraalCompiler.emitBackEnd(GraalCompiler.java:275) at org.graalvm.compiler.core.GraalCompiler.compile(GraalCompiler.java:175) at org.graalvm.compiler.core.GraalCompiler.compileGraph(GraalCompiler.java:160) at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compilePEGraph(TruffleCompilerImpl.java:445) at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compileAST(TruffleCompilerImpl.java:391) at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:544) at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:493) at org.graalvm.compiler.core.CompilationWrapper.run(CompilationWrapper.java:167) at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.doCompile(TruffleCompilerImpl.java:222) at org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime.doCompile(GraalTruffleRuntime.java:679) at org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime$1.run(GraalTruffleRuntime.java:745) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) To disable compilation failure notifications, set CompilationFailureAction to Silent (e.g., -Dgraal.CompilationFailureAction=Silent). To print a message for a compilation failure without retrying the compilation, set CompilationFailureAction to Print (e.g., -Dgraal.CompilationFailureAction=Print). Retrying compilation of BFRepeatingNode at 222b298d From aph at redhat.com Tue May 1 17:09:52 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 1 May 2018 18:09:52 +0100 Subject: Can't run Graal with JDK 11 Message-ID: <5649762d-38e2-f857-d36b-74ccc473366f@redhat.com> I get this when trying to AOT-compile using latest jdk-jdk and latest Graal: Exception in thread "main" java.lang.NoClassDefFoundError: jdk/internal/vm/compiler/word/WordBase at jdk.aot/jdk.tools.jaotc.GraalFilters.getSpecialArgumentAndReturnTypes(GraalFilters.java:112) at jdk.aot/jdk.tools.jaotc.GraalFilters.(GraalFilters.java:119) at jdk.aot/jdk.tools.jaotc.Main.run(Main.java:148) at jdk.aot/jdk.tools.jaotc.Main.run(Main.java:101) at jdk.aot/jdk.tools.jaotc.Main.main(Main.java:80) Caused by: java.lang.ClassNotFoundException: jdk.internal.vm.compiler.word.WordBase I think the problem is that jdk/internal/vm/compiler/word/WordBase is in the module jdk.internal.vm.compiler, and Graal's "mx vm" command does -upgrade-module-path=/local/graal/graal/compiler/mxbuild/modules/jdk.internal.vm.compiler.jar:/local/graal/graal/compiler/mxbuild/modules/jdk.internal.vm.compiler.management.jar Unfortunately, jdk/internal/vm/compiler/word/WordBase is only present in OpenJDK's version of the jdk.internal.vm.compiler module, and is not present in Graal's jdk.internal.vm.compiler. Suggestions? -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From vladimir.kozlov at oracle.com Tue May 1 17:58:59 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 1 May 2018 10:58:59 -0700 Subject: RFR(XL) 8199755: Update Graal In-Reply-To: <8a6a744e-c36d-ddd4-2250-a8b19a7c3da3@redhat.com> References: <306f4fea-68f1-473d-a092-47b0703b7088@oracle.com> <8a6a744e-c36d-ddd4-2250-a8b19a7c3da3@redhat.com> Message-ID: Hi Andrew, I see that your AOT Graal changes were pushed 2 days ago: https://github.com/oracle/graal/commit/f4f72334449373627070999e9c81ffc5cf1cb067 "8199755: Update Graal" was pushed *week* ago: http://hg.openjdk.java.net/jdk/jdk/rev/26ebfe8ce852 I don't think it could be called "missed". We can sync Graal into OpenJDK again this week too. But we have to resolve few issues we found with last sync - Graal *dropped CMS GC* support and all our AOT tests and Graal JIT testing fail now with CMS. I fixed 8202273 and now working on 8184349. This is shared code and your AOT implementation for Aarch64 should not be affected (I hope). Regards, Vladimir On 5/1/18 10:22 AM, Andrew Haley wrote: > On 04/23/2018 09:52 PM, dean.long at oracle.com wrote: >> See the bug for the full list of Graal changes. >> I separated out the changes needed for module-info.java and AOT. > > My AArch64 AOT changes seem to have missed the boat. I can't tell you > how disappointed I am after all this work. :-( > > Did you really push Graal to OpenJDK just before importing my AOT > changes into Graal? Can we have another Graal import, please? > From doug.simon at oracle.com Tue May 1 18:45:49 2018 From: doug.simon at oracle.com (Doug Simon) Date: Tue, 1 May 2018 20:45:49 +0200 Subject: Can't run Graal with JDK 11 In-Reply-To: <5649762d-38e2-f857-d36b-74ccc473366f@redhat.com> References: <5649762d-38e2-f857-d36b-74ccc473366f@redhat.com> Message-ID: > On 1 May 2018, at 19:09, Andrew Haley wrote: > > I get this when trying to AOT-compile using latest jdk-jdk and latest > Graal: > > Exception in thread "main" java.lang.NoClassDefFoundError: jdk/internal/vm/compiler/word/WordBase > at jdk.aot/jdk.tools.jaotc.GraalFilters.getSpecialArgumentAndReturnTypes(GraalFilters.java:112) > at jdk.aot/jdk.tools.jaotc.GraalFilters.(GraalFilters.java:119) > at jdk.aot/jdk.tools.jaotc.Main.run(Main.java:148) > at jdk.aot/jdk.tools.jaotc.Main.run(Main.java:101) > at jdk.aot/jdk.tools.jaotc.Main.main(Main.java:80) > Caused by: java.lang.ClassNotFoundException: jdk.internal.vm.compiler.word.WordBase > > I think the problem is that jdk/internal/vm/compiler/word/WordBase is > in the module jdk.internal.vm.compiler, and Graal's "mx vm" command does > > -upgrade-module-path=/local/graal/graal/compiler/mxbuild/modules/jdk.internal.vm.compiler.jar:/local/graal/graal/compiler/mxbuild/modules/jdk.internal.vm.compiler.management.jar > > Unfortunately, jdk/internal/vm/compiler/word/WordBase is only present > in OpenJDK's version of the jdk.internal.vm.compiler module, and is > not present in Graal's jdk.internal.vm.compiler. > > Suggestions? It sounds like you are somehow running jaotc via mx. I'm not sure how this is even possible. As you can see, it doesn't work. If not, what mx command are you running to produce this error? -Doug From josef.eisl at jku.at Wed May 2 07:37:30 2018 From: josef.eisl at jku.at (Josef Eisl) Date: Wed, 2 May 2018 09:37:30 +0200 Subject: OutOfMemoryError on my BF interpreter In-Reply-To: References: Message-ID: <6435958d-15ab-aef0-f499-96ba655e5fa0@jku.at> Hi Cristian! Thanks for your report. We have very recently merged a change to mitigate the OOME in this part of linear scan [1]. Unfortunately this was after the release so it is not included in 1.0.0 RC1. You would need to build your own GraalVM from source to see if the change helps. That said, the root cause of the OOME is usually a huge compilation unit thrown into the compiler. For more details, see also the discussion on the OpenJDK bug tracker [2]. The reasons for big compilation units can me manifold. In case of Truffle, the issues can often be solved by adding TruffleBoundaries [3], i.e., instructing the compiler not to inline specified methods into the truffle graph. Maybe a Truffle expert can give you more hints on that. Hope that helps, Josef [1]: https://github.com/oracle/graal/commit/c0a15c562daa0338f61a6e7c22476cf145af5a66 [2]: https://bugs.openjdk.java.net/browse/JDK-8199890 [3]: http://www.graalvm.org/truffle/javadoc/com/oracle/truffle/api/CompilerDirectives.TruffleBoundary.html On 01/05/18 08:24, Cristian Esquivias wrote: > With the release of 1.0.0-RC1 I thought I'd come back and see how > Graal is doing. Great job; Graal is really starting to look like the > platform everyone should pay attention to. > > To get back into it, I built a Brainf*ck interpreter with Truffle > (https://github.com/cesquivias/bf-graal). I modeled it after > simplelanguage so it should be easy to build and run. > > While running the mandelbrot program, about halfway through the > program, the VM starts throwing OutOfMemoryError exceptions (one > pasted below). It created a zip file under dumps/ but it contains a > .bgv file I don't what to do with and a log file that just contains > the stacktrace. > > I ran jvisualvm and took a heap dump. There are ~200k long[] objects > alone created by graal taking up ~50% of the memory. Digging through > the references it seems to be created/retained in > > org.graalvm.compiler.lir.alloc.lsra.LinearScanLifetimeAnalysisPhase$$Lambda$48#1 > [GC root - Java frame] > > I'd provide some file from jvisualvm but the save button is grayed > out. I hope this is enough info. My interpreter is up on GitHub so > feel free to test it out. > > I tested this both on the community & enterprise edition on Ubuntu > 18.04 in VirtualBox. > > - Cristian > > > Thread[TruffleCompilerThread-12,10,main]: Compilation of > BFRepeatingNode at 222b298d failed: java.lang.OutOfMemoryError: Java > heap space > at org.graalvm.compiler.lir.util.IndexedValueMap.(IndexedValueMap.java:55) > at org.graalvm.compiler.lir.dfa.RegStackValueSet.(RegStackValueSet.java:62) > at org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:70) > at org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:46) > at org.graalvm.compiler.lir.dfa.LocationMarker.processBlock(LocationMarker.java:107) > at org.graalvm.compiler.lir.dfa.LocationMarker.build(LocationMarker.java:81) > at org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:51) > at org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:47) > at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) > at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) > at org.graalvm.compiler.lir.phases.LIRPhaseSuite.run(LIRPhaseSuite.java:96) > at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) > at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) > at org.graalvm.compiler.core.GraalCompiler.emitLowLevel(GraalCompiler.java:367) > at org.graalvm.compiler.core.GraalCompiler.emitLIR0(GraalCompiler.java:336) > at org.graalvm.compiler.core.GraalCompiler.emitLIR(GraalCompiler.java:295) > at org.graalvm.compiler.core.GraalCompiler.emitBackEnd(GraalCompiler.java:275) > at org.graalvm.compiler.core.GraalCompiler.compile(GraalCompiler.java:175) > at org.graalvm.compiler.core.GraalCompiler.compileGraph(GraalCompiler.java:160) > at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compilePEGraph(TruffleCompilerImpl.java:445) > at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compileAST(TruffleCompilerImpl.java:391) > at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:544) > at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:493) > at org.graalvm.compiler.core.CompilationWrapper.run(CompilationWrapper.java:167) > at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.doCompile(TruffleCompilerImpl.java:222) > at org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime.doCompile(GraalTruffleRuntime.java:679) > at org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime$1.run(GraalTruffleRuntime.java:745) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > To disable compilation failure notifications, set > CompilationFailureAction to Silent (e.g., > -Dgraal.CompilationFailureAction=Silent). > To print a message for a compilation failure without retrying the > compilation, set CompilationFailureAction to Print (e.g., > -Dgraal.CompilationFailureAction=Print). > Retrying compilation of BFRepeatingNode at 222b298d > From aph at redhat.com Wed May 2 07:41:05 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 2 May 2018 08:41:05 +0100 Subject: Can't run Graal with JDK 11 In-Reply-To: References: <5649762d-38e2-f857-d36b-74ccc473366f@redhat.com> Message-ID: <21ea33fb-57ad-6c13-0f10-b727cdc8fa2e@redhat.com> On 05/01/2018 07:45 PM, Doug Simon wrote: > It sounds like you are somehow running jaotc via mx. I'm not sure how this is even possible. As you can see, it doesn't work. It worked until a few days ago. It's the only way I have to test aot. So this is a regression. Given that "it doesn't work", how is anyone supposed to get any work done? > If not, what mx command are you running to produce this error? Same as before: jaotc -J--module-path=/local/graal/graal/sdk/mxbuild/modules/org.graalvm.graal_sdk.jar:/local/graal/graal/truffle/mxbuild/modules/com.oracle.truffle.truffle_api.jar:build/linux-aarch64-normal-server-slowdebug/jdk/modules/jdk.internal.vm.compiler \ -J--upgrade-module-path=/local/graal/graal/compiler/mxbuild/modules/jdk.internal.vm.compiler.jar:/local/graal/graal/compiler/mxbuild/modules/jdk.internal.vm.compiler.management.jar But because of this recent refactoring it's no longer possible to to use --upgrade-module-path. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From cristian.esquivias at gmail.com Wed May 2 08:46:52 2018 From: cristian.esquivias at gmail.com (Cristian Esquivias) Date: Wed, 2 May 2018 01:46:52 -0700 Subject: OutOfMemoryError on my BF interpreter In-Reply-To: <6435958d-15ab-aef0-f499-96ba655e5fa0@jku.at> References: <6435958d-15ab-aef0-f499-96ba655e5fa0@jku.at> Message-ID: Hi Josef, Thanks for the context. I'll try to build the GraalVM and see if it solves the issue. After reading the bug report, here's a little bit more info that I hope helps: * The report says the SSA register allocator could be the problem on "large inputs". I'm not sure what constitutes a large input, but my mandelbrot script creates around 11,000 nodes in total. That's probably not a huge amount, but they're all within one function since BF doesn't have the concept of functions. That's probably a little more unusual. * My interpreter doesn't put pressure on the heap. After an initial allocation of a 1,000-10,000 size int array no more allocations are done for the rest of a script's life. This is proven out by jvisualvm. The heap stays flat throughout most of the run and only spikes right before the OOME occurs. * There are very few (essentially no) opportunities for adding TruffleBoundary annotations since nearly all my nodes' execute methods are one line that do little more than fetch an item out of my array and inc/dec the value. The only calls out to external methods are calls to VirtualFrame.get [1]/set [2] some variable values, calls to stdout's print method [3] and stdin's read method [4]. A truffle expert's help would definitely be appreciated here since I'm just making an educated guess. * The OOME occurs inside a loop node. Could this OOME be caused by loop unrolling [5]? Thanks, Cristian [1]: https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/DecDataNode.java [2]: https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/DecPointerNode.java [3]: https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/PrintDataNode.java [4]: https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/ReadDataNode.java [5]: https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/BFRepeatingNode.java On Wed, May 2, 2018 at 12:37 AM, Josef Eisl wrote: > Hi Cristian! > > Thanks for your report. We have very recently merged a change to > mitigate the OOME in this part of linear scan [1]. Unfortunately this > was after the release so it is not included in 1.0.0 RC1. You would need > to build your own GraalVM from source to see if the change helps. > > That said, the root cause of the OOME is usually a huge compilation unit > thrown into the compiler. For more details, see also the discussion on > the OpenJDK bug tracker [2]. > The reasons for big compilation units can me manifold. In case of > Truffle, the issues can often be solved by adding TruffleBoundaries [3], > i.e., instructing the compiler not to inline specified methods into the > truffle graph. Maybe a Truffle expert can give you more hints on that. > > Hope that helps, > > Josef > > [1]: > https://github.com/oracle/graal/commit/c0a15c562daa0338f61a6e7c22476cf145af5a66 > [2]: https://bugs.openjdk.java.net/browse/JDK-8199890 > [3]: > http://www.graalvm.org/truffle/javadoc/com/oracle/truffle/api/CompilerDirectives.TruffleBoundary.html > > On 01/05/18 08:24, Cristian Esquivias wrote: >> With the release of 1.0.0-RC1 I thought I'd come back and see how >> Graal is doing. Great job; Graal is really starting to look like the >> platform everyone should pay attention to. >> >> To get back into it, I built a Brainf*ck interpreter with Truffle >> (https://github.com/cesquivias/bf-graal). I modeled it after >> simplelanguage so it should be easy to build and run. >> >> While running the mandelbrot program, about halfway through the >> program, the VM starts throwing OutOfMemoryError exceptions (one >> pasted below). It created a zip file under dumps/ but it contains a >> .bgv file I don't what to do with and a log file that just contains >> the stacktrace. >> >> I ran jvisualvm and took a heap dump. There are ~200k long[] objects >> alone created by graal taking up ~50% of the memory. Digging through >> the references it seems to be created/retained in >> >> org.graalvm.compiler.lir.alloc.lsra.LinearScanLifetimeAnalysisPhase$$Lambda$48#1 >> [GC root - Java frame] >> >> I'd provide some file from jvisualvm but the save button is grayed >> out. I hope this is enough info. My interpreter is up on GitHub so >> feel free to test it out. >> >> I tested this both on the community & enterprise edition on Ubuntu >> 18.04 in VirtualBox. >> >> - Cristian >> >> >> Thread[TruffleCompilerThread-12,10,main]: Compilation of >> BFRepeatingNode at 222b298d failed: java.lang.OutOfMemoryError: Java >> heap space >> at org.graalvm.compiler.lir.util.IndexedValueMap.(IndexedValueMap.java:55) >> at org.graalvm.compiler.lir.dfa.RegStackValueSet.(RegStackValueSet.java:62) >> at org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:70) >> at org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:46) >> at org.graalvm.compiler.lir.dfa.LocationMarker.processBlock(LocationMarker.java:107) >> at org.graalvm.compiler.lir.dfa.LocationMarker.build(LocationMarker.java:81) >> at org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:51) >> at org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:47) >> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) >> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) >> at org.graalvm.compiler.lir.phases.LIRPhaseSuite.run(LIRPhaseSuite.java:96) >> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) >> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) >> at org.graalvm.compiler.core.GraalCompiler.emitLowLevel(GraalCompiler.java:367) >> at org.graalvm.compiler.core.GraalCompiler.emitLIR0(GraalCompiler.java:336) >> at org.graalvm.compiler.core.GraalCompiler.emitLIR(GraalCompiler.java:295) >> at org.graalvm.compiler.core.GraalCompiler.emitBackEnd(GraalCompiler.java:275) >> at org.graalvm.compiler.core.GraalCompiler.compile(GraalCompiler.java:175) >> at org.graalvm.compiler.core.GraalCompiler.compileGraph(GraalCompiler.java:160) >> at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compilePEGraph(TruffleCompilerImpl.java:445) >> at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compileAST(TruffleCompilerImpl.java:391) >> at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:544) >> at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:493) >> at org.graalvm.compiler.core.CompilationWrapper.run(CompilationWrapper.java:167) >> at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.doCompile(TruffleCompilerImpl.java:222) >> at org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime.doCompile(GraalTruffleRuntime.java:679) >> at org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime$1.run(GraalTruffleRuntime.java:745) >> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >> at java.lang.Thread.run(Thread.java:748) >> To disable compilation failure notifications, set >> CompilationFailureAction to Silent (e.g., >> -Dgraal.CompilationFailureAction=Silent). >> To print a message for a compilation failure without retrying the >> compilation, set CompilationFailureAction to Print (e.g., >> -Dgraal.CompilationFailureAction=Print). >> Retrying compilation of BFRepeatingNode at 222b298d >> > From doug.simon at oracle.com Wed May 2 08:55:06 2018 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 2 May 2018 10:55:06 +0200 Subject: Can't run Graal with JDK 11 In-Reply-To: <21ea33fb-57ad-6c13-0f10-b727cdc8fa2e@redhat.com> References: <5649762d-38e2-f857-d36b-74ccc473366f@redhat.com> <21ea33fb-57ad-6c13-0f10-b727cdc8fa2e@redhat.com> Message-ID: Hi Andrew, Sorry, I misunderstood that you were giving "mx vm" as an example of how you run jaotc. The underlying problem here is that GitHub Graal contains packages such as org.graalvm.collections. Until recently, these classes were only used by Graal. However, they are now being used by Truffle which is not part of the Graal JDK module (and probably never will be). Furthermore, the Graal JDK module does not unqualified export these packages which means that Truffle apps on the class path cannot see them: java.lang.IllegalAccessError: class com.oracle.truffle.api.frame.FrameDescriptor (in unnamed module @0x16e7dcfd) cannot access class org.graalvm.collections.EconomicMap (in module jdk.internal.vm.compiler) because module jdk.internal.vm.compiler does not export org.graalvm.collections to unnamed module @0x16e7dcfd at com.oracle.truffle.api.frame.FrameDescriptor.(FrameDescriptor.java:89) at com.oracle.truffle.api.frame.FrameDescriptor.(FrameDescriptor.java:76) The solution adopted is to rename these "shared" packages when they are copied into OpenJDK Graal[1]. This now creates the problems you are seeing. The last time we discussed this with the HotSpot compiler team, the proposal was to move jaotc into GitHub Graal and have the copy in OpenJDK be effectively read-only (like Graal currently is). I'm not sure of the status implementing this proposal - someone from the HotSpot compiler team should comment on that. Until that's done, I agree that getting work done on jaotc will be challenging. The only workaround I can think of is to use the `mx updategraalinopenjdk` command as part of your workflow. -Doug [1] https://github.com/oracle/graal/blob/master/compiler/mx.compiler/mx_compiler.py#L1126 > On 2 May 2018, at 09:41, Andrew Haley wrote: > > On 05/01/2018 07:45 PM, Doug Simon wrote: >> It sounds like you are somehow running jaotc via mx. I'm not sure how this is even possible. As you can see, it doesn't work. > > It worked until a few days ago. It's the only way I have to test aot. > So this is a regression. Given that "it doesn't work", how is anyone > supposed to get any work done? > >> If not, what mx command are you running to produce this error? > > Same as before: > > jaotc -J--module-path=/local/graal/graal/sdk/mxbuild/modules/org.graalvm.graal_sdk.jar:/local/graal/graal/truffle/mxbuild/modules/com.oracle.truffle.truffle_api.jar:build/linux-aarch64-normal-server-slowdebug/jdk/modules/jdk.internal.vm.compiler \ > -J--upgrade-module-path=/local/graal/graal/compiler/mxbuild/modules/jdk.internal.vm.compiler.jar:/local/graal/graal/compiler/mxbuild/modules/jdk.internal.vm.compiler.management.jar > > But because of this recent refactoring it's no longer possible to to > use --upgrade-module-path. > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Wed May 2 09:08:30 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 2 May 2018 10:08:30 +0100 Subject: Can't run Graal with JDK 11 In-Reply-To: References: <5649762d-38e2-f857-d36b-74ccc473366f@redhat.com> <21ea33fb-57ad-6c13-0f10-b727cdc8fa2e@redhat.com> Message-ID: <81c800e0-2f7f-126e-b646-6e4465d4ee5a@redhat.com> On 05/02/2018 09:55 AM, Doug Simon wrote: > Until that's done, I agree that getting work done on jaotc will be > challenging. The only workaround I can think of is to use the `mx > updategraalinopenjdk` command as part of your workflow. That sounds like it would probably work, but of course it corrupts the source of the OpenJDK I'm working on. I guess I can live with that for now, but as Graal is part of OpenJDK and seems likely to be important we're going to have to make it easier for people to work on. In the long run, as you might expect, I'd prefer it to be a first-class part of OpenJDK. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From thomas.wuerthinger at oracle.com Wed May 2 09:09:07 2018 From: thomas.wuerthinger at oracle.com (Thomas Wuerthinger) Date: Wed, 2 May 2018 11:09:07 +0200 Subject: OutOfMemoryError on my BF interpreter In-Reply-To: References: <6435958d-15ab-aef0-f499-96ba655e5fa0@jku.at> Message-ID: <364A3D3B-EE37-4610-BA74-313FCDD7486F@oracle.com> One cause could be excessive escape analysis of the Frame. How many local variables do you have? Can you provide us a command line / input so we can reproduce? Thanks, thomas > On 2 May 2018, at 10:46, Cristian Esquivias wrote: > > Hi Josef, > > Thanks for the context. I'll try to build the GraalVM and see if it > solves the issue. > > After reading the bug report, here's a little bit more info that I hope helps: > > * The report says the SSA register allocator could be the problem on > "large inputs". I'm not sure what constitutes a large input, but my > mandelbrot script creates around 11,000 nodes in total. That's > probably not a huge amount, but they're all within one function since > BF doesn't have the concept of functions. That's probably a little > more unusual. > * My interpreter doesn't put pressure on the heap. After an initial > allocation of a 1,000-10,000 size int array no more allocations are > done for the rest of a script's life. This is proven out by jvisualvm. > The heap stays flat throughout most of the run and only spikes right > before the OOME occurs. > * There are very few (essentially no) opportunities for adding > TruffleBoundary annotations since nearly all my nodes' execute methods > are one line that do little more than fetch an item out of my array > and inc/dec the value. The only calls out to external methods are > calls to VirtualFrame.get [1]/set [2] some variable values, calls to > stdout's print method [3] and stdin's read method [4]. A truffle > expert's help would definitely be appreciated here since I'm just > making an educated guess. > * The OOME occurs inside a loop node. Could this OOME be caused by > loop unrolling [5]? > > Thanks, > Cristian > > [1]: https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/DecDataNode.java > [2]: https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/DecPointerNode.java > [3]: https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/PrintDataNode.java > [4]: https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/ReadDataNode.java > [5]: https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/BFRepeatingNode.java > > On Wed, May 2, 2018 at 12:37 AM, Josef Eisl wrote: >> Hi Cristian! >> >> Thanks for your report. We have very recently merged a change to >> mitigate the OOME in this part of linear scan [1]. Unfortunately this >> was after the release so it is not included in 1.0.0 RC1. You would need >> to build your own GraalVM from source to see if the change helps. >> >> That said, the root cause of the OOME is usually a huge compilation unit >> thrown into the compiler. For more details, see also the discussion on >> the OpenJDK bug tracker [2]. >> The reasons for big compilation units can me manifold. In case of >> Truffle, the issues can often be solved by adding TruffleBoundaries [3], >> i.e., instructing the compiler not to inline specified methods into the >> truffle graph. Maybe a Truffle expert can give you more hints on that. >> >> Hope that helps, >> >> Josef >> >> [1]: >> https://github.com/oracle/graal/commit/c0a15c562daa0338f61a6e7c22476cf145af5a66 >> [2]: https://bugs.openjdk.java.net/browse/JDK-8199890 >> [3]: >> http://www.graalvm.org/truffle/javadoc/com/oracle/truffle/api/CompilerDirectives.TruffleBoundary.html >> >> On 01/05/18 08:24, Cristian Esquivias wrote: >>> With the release of 1.0.0-RC1 I thought I'd come back and see how >>> Graal is doing. Great job; Graal is really starting to look like the >>> platform everyone should pay attention to. >>> >>> To get back into it, I built a Brainf*ck interpreter with Truffle >>> (https://github.com/cesquivias/bf-graal). I modeled it after >>> simplelanguage so it should be easy to build and run. >>> >>> While running the mandelbrot program, about halfway through the >>> program, the VM starts throwing OutOfMemoryError exceptions (one >>> pasted below). It created a zip file under dumps/ but it contains a >>> .bgv file I don't what to do with and a log file that just contains >>> the stacktrace. >>> >>> I ran jvisualvm and took a heap dump. There are ~200k long[] objects >>> alone created by graal taking up ~50% of the memory. Digging through >>> the references it seems to be created/retained in >>> >>> org.graalvm.compiler.lir.alloc.lsra.LinearScanLifetimeAnalysisPhase$$Lambda$48#1 >>> [GC root - Java frame] >>> >>> I'd provide some file from jvisualvm but the save button is grayed >>> out. I hope this is enough info. My interpreter is up on GitHub so >>> feel free to test it out. >>> >>> I tested this both on the community & enterprise edition on Ubuntu >>> 18.04 in VirtualBox. >>> >>> - Cristian >>> >>> >>> Thread[TruffleCompilerThread-12,10,main]: Compilation of >>> BFRepeatingNode at 222b298d failed: java.lang.OutOfMemoryError: Java >>> heap space >>> at org.graalvm.compiler.lir.util.IndexedValueMap.(IndexedValueMap.java:55) >>> at org.graalvm.compiler.lir.dfa.RegStackValueSet.(RegStackValueSet.java:62) >>> at org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:70) >>> at org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:46) >>> at org.graalvm.compiler.lir.dfa.LocationMarker.processBlock(LocationMarker.java:107) >>> at org.graalvm.compiler.lir.dfa.LocationMarker.build(LocationMarker.java:81) >>> at org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:51) >>> at org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:47) >>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) >>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) >>> at org.graalvm.compiler.lir.phases.LIRPhaseSuite.run(LIRPhaseSuite.java:96) >>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) >>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) >>> at org.graalvm.compiler.core.GraalCompiler.emitLowLevel(GraalCompiler.java:367) >>> at org.graalvm.compiler.core.GraalCompiler.emitLIR0(GraalCompiler.java:336) >>> at org.graalvm.compiler.core.GraalCompiler.emitLIR(GraalCompiler.java:295) >>> at org.graalvm.compiler.core.GraalCompiler.emitBackEnd(GraalCompiler.java:275) >>> at org.graalvm.compiler.core.GraalCompiler.compile(GraalCompiler.java:175) >>> at org.graalvm.compiler.core.GraalCompiler.compileGraph(GraalCompiler.java:160) >>> at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compilePEGraph(TruffleCompilerImpl.java:445) >>> at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compileAST(TruffleCompilerImpl.java:391) >>> at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:544) >>> at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:493) >>> at org.graalvm.compiler.core.CompilationWrapper.run(CompilationWrapper.java:167) >>> at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.doCompile(TruffleCompilerImpl.java:222) >>> at org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime.doCompile(GraalTruffleRuntime.java:679) >>> at org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime$1.run(GraalTruffleRuntime.java:745) >>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >>> at java.lang.Thread.run(Thread.java:748) >>> To disable compilation failure notifications, set >>> CompilationFailureAction to Silent (e.g., >>> -Dgraal.CompilationFailureAction=Silent). >>> To print a message for a compilation failure without retrying the >>> compilation, set CompilationFailureAction to Print (e.g., >>> -Dgraal.CompilationFailureAction=Print). >>> Retrying compilation of BFRepeatingNode at 222b298d >>> >> From doug.simon at oracle.com Wed May 2 09:18:21 2018 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 2 May 2018 11:18:21 +0200 Subject: Can't run Graal with JDK 11 In-Reply-To: <81c800e0-2f7f-126e-b646-6e4465d4ee5a@redhat.com> References: <5649762d-38e2-f857-d36b-74ccc473366f@redhat.com> <21ea33fb-57ad-6c13-0f10-b727cdc8fa2e@redhat.com> <81c800e0-2f7f-126e-b646-6e4465d4ee5a@redhat.com> Message-ID: <787ADE77-4C71-4B21-A0AD-110C5ABB7858@oracle.com> > On 2 May 2018, at 11:08, Andrew Haley wrote: > > On 05/02/2018 09:55 AM, Doug Simon wrote: > >> Until that's done, I agree that getting work done on jaotc will be >> challenging. The only workaround I can think of is to use the `mx >> updategraalinopenjdk` command as part of your workflow. > > That sounds like it would probably work, but of course it corrupts the > source of the OpenJDK I'm working on. The updategraalinopenjdk tool should only touch the Graal directories in OpenJDK. If not, it's a bug and will be fixed. > I guess I can live with that > for now, but as Graal is part of OpenJDK and seems likely to be > important we're going to have to make it easier for people to work on. > In the long run, as you might expect, I'd prefer it to be a > first-class part of OpenJDK. Yes, I agree that would simplify the OpenJDK development process. However, Graal is used in a number of other contexts and as such needs to be developed and tested independently of OpenJDK. We will continue to try mitigate OpenJDK development pain by providing tools such as updategraalinopenjdk. -Doug > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From boris.spasojevic at oracle.com Wed May 2 09:18:51 2018 From: boris.spasojevic at oracle.com (Boris Spasojevic) Date: Wed, 2 May 2018 11:18:51 +0200 Subject: OutOfMemoryError on my BF interpreter In-Reply-To: References: <6435958d-15ab-aef0-f499-96ba655e5fa0@jku.at> Message-ID: Hey Christian, This could very well be caused by the calls to System.out.print and System.in.read. Could you please try wrapping calls to such external methods and adding a @TruffleBoundary annotation? The end goal would be something like: public class PrintDataNode extends BFNode { private final FrameSlot ptr; private final FrameSlot data; public PrintDataNode(FrameSlot ptr, FrameSlot data) { this.ptr = ptr; this.data = data; } @TruffleBoundary void print(char c) { System.out.print(c); } @Override public void execute(VirtualFrame frame) { try { print((char) ((byte[]) frame.getObject(data))[frame.getInt(ptr)]); } catch (FrameSlotTypeException e) { e.printStackTrace(); } } } You can see a similar pattern being used in Simple language (https://github.com/graalvm/simplelanguage/blob/master/language/src/main/java/com/oracle/truffle/sl/builtins/SLPrintlnBuiltin.java) Hope that helps! BoriS On 05/02/2018 10:46 AM, Cristian Esquivias wrote: > Hi Josef, > > Thanks for the context. I'll try to build the GraalVM and see if it > solves the issue. > > After reading the bug report, here's a little bit more info that I hope helps: > > * The report says the SSA register allocator could be the problem on > "large inputs". I'm not sure what constitutes a large input, but my > mandelbrot script creates around 11,000 nodes in total. That's > probably not a huge amount, but they're all within one function since > BF doesn't have the concept of functions. That's probably a little > more unusual. > * My interpreter doesn't put pressure on the heap. After an initial > allocation of a 1,000-10,000 size int array no more allocations are > done for the rest of a script's life. This is proven out by jvisualvm. > The heap stays flat throughout most of the run and only spikes right > before the OOME occurs. > * There are very few (essentially no) opportunities for adding > TruffleBoundary annotations since nearly all my nodes' execute methods > are one line that do little more than fetch an item out of my array > and inc/dec the value. The only calls out to external methods are > calls to VirtualFrame.get [1]/set [2] some variable values, calls to > stdout's print method [3] and stdin's read method [4]. A truffle > expert's help would definitely be appreciated here since I'm just > making an educated guess. > * The OOME occurs inside a loop node. Could this OOME be caused by > loop unrolling [5]? > > Thanks, > Cristian > > [1]: https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/DecDataNode.java > [2]: https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/DecPointerNode.java > [3]: https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/PrintDataNode.java > [4]: https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/ReadDataNode.java > [5]: https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/BFRepeatingNode.java > > On Wed, May 2, 2018 at 12:37 AM, Josef Eisl wrote: >> Hi Cristian! >> >> Thanks for your report. We have very recently merged a change to >> mitigate the OOME in this part of linear scan [1]. Unfortunately this >> was after the release so it is not included in 1.0.0 RC1. You would need >> to build your own GraalVM from source to see if the change helps. >> >> That said, the root cause of the OOME is usually a huge compilation unit >> thrown into the compiler. For more details, see also the discussion on >> the OpenJDK bug tracker [2]. >> The reasons for big compilation units can me manifold. In case of >> Truffle, the issues can often be solved by adding TruffleBoundaries [3], >> i.e., instructing the compiler not to inline specified methods into the >> truffle graph. Maybe a Truffle expert can give you more hints on that. >> >> Hope that helps, >> >> Josef >> >> [1]: >> https://github.com/oracle/graal/commit/c0a15c562daa0338f61a6e7c22476cf145af5a66 >> [2]: https://bugs.openjdk.java.net/browse/JDK-8199890 >> [3]: >> http://www.graalvm.org/truffle/javadoc/com/oracle/truffle/api/CompilerDirectives.TruffleBoundary.html >> >> On 01/05/18 08:24, Cristian Esquivias wrote: >>> With the release of 1.0.0-RC1 I thought I'd come back and see how >>> Graal is doing. Great job; Graal is really starting to look like the >>> platform everyone should pay attention to. >>> >>> To get back into it, I built a Brainf*ck interpreter with Truffle >>> (https://github.com/cesquivias/bf-graal). I modeled it after >>> simplelanguage so it should be easy to build and run. >>> >>> While running the mandelbrot program, about halfway through the >>> program, the VM starts throwing OutOfMemoryError exceptions (one >>> pasted below). It created a zip file under dumps/ but it contains a >>> .bgv file I don't what to do with and a log file that just contains >>> the stacktrace. >>> >>> I ran jvisualvm and took a heap dump. There are ~200k long[] objects >>> alone created by graal taking up ~50% of the memory. Digging through >>> the references it seems to be created/retained in >>> >>> org.graalvm.compiler.lir.alloc.lsra.LinearScanLifetimeAnalysisPhase$$Lambda$48#1 >>> [GC root - Java frame] >>> >>> I'd provide some file from jvisualvm but the save button is grayed >>> out. I hope this is enough info. My interpreter is up on GitHub so >>> feel free to test it out. >>> >>> I tested this both on the community & enterprise edition on Ubuntu >>> 18.04 in VirtualBox. >>> >>> - Cristian >>> >>> >>> Thread[TruffleCompilerThread-12,10,main]: Compilation of >>> BFRepeatingNode at 222b298d failed: java.lang.OutOfMemoryError: Java >>> heap space >>> at org.graalvm.compiler.lir.util.IndexedValueMap.(IndexedValueMap.java:55) >>> at org.graalvm.compiler.lir.dfa.RegStackValueSet.(RegStackValueSet.java:62) >>> at org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:70) >>> at org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:46) >>> at org.graalvm.compiler.lir.dfa.LocationMarker.processBlock(LocationMarker.java:107) >>> at org.graalvm.compiler.lir.dfa.LocationMarker.build(LocationMarker.java:81) >>> at org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:51) >>> at org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:47) >>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) >>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) >>> at org.graalvm.compiler.lir.phases.LIRPhaseSuite.run(LIRPhaseSuite.java:96) >>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) >>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) >>> at org.graalvm.compiler.core.GraalCompiler.emitLowLevel(GraalCompiler.java:367) >>> at org.graalvm.compiler.core.GraalCompiler.emitLIR0(GraalCompiler.java:336) >>> at org.graalvm.compiler.core.GraalCompiler.emitLIR(GraalCompiler.java:295) >>> at org.graalvm.compiler.core.GraalCompiler.emitBackEnd(GraalCompiler.java:275) >>> at org.graalvm.compiler.core.GraalCompiler.compile(GraalCompiler.java:175) >>> at org.graalvm.compiler.core.GraalCompiler.compileGraph(GraalCompiler.java:160) >>> at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compilePEGraph(TruffleCompilerImpl.java:445) >>> at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compileAST(TruffleCompilerImpl.java:391) >>> at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:544) >>> at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:493) >>> at org.graalvm.compiler.core.CompilationWrapper.run(CompilationWrapper.java:167) >>> at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.doCompile(TruffleCompilerImpl.java:222) >>> at org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime.doCompile(GraalTruffleRuntime.java:679) >>> at org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime$1.run(GraalTruffleRuntime.java:745) >>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >>> at java.lang.Thread.run(Thread.java:748) >>> To disable compilation failure notifications, set >>> CompilationFailureAction to Silent (e.g., >>> -Dgraal.CompilationFailureAction=Silent). >>> To print a message for a compilation failure without retrying the >>> compilation, set CompilationFailureAction to Print (e.g., >>> -Dgraal.CompilationFailureAction=Print). >>> Retrying compilation of BFRepeatingNode at 222b298d >>> From aph at redhat.com Wed May 2 10:01:56 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 2 May 2018 11:01:56 +0100 Subject: Can't run Graal with JDK 11 In-Reply-To: <787ADE77-4C71-4B21-A0AD-110C5ABB7858@oracle.com> References: <5649762d-38e2-f857-d36b-74ccc473366f@redhat.com> <21ea33fb-57ad-6c13-0f10-b727cdc8fa2e@redhat.com> <81c800e0-2f7f-126e-b646-6e4465d4ee5a@redhat.com> <787ADE77-4C71-4B21-A0AD-110C5ABB7858@oracle.com> Message-ID: On 05/02/2018 10:18 AM, Doug Simon wrote: > > >> On 2 May 2018, at 11:08, Andrew Haley wrote: >> >> On 05/02/2018 09:55 AM, Doug Simon wrote: >> >>> Until that's done, I agree that getting work done on jaotc will be >>> challenging. The only workaround I can think of is to use the `mx >>> updategraalinopenjdk` command as part of your workflow. >> >> That sounds like it would probably work, but of course it corrupts the >> source of the OpenJDK I'm working on. > > The updategraalinopenjdk tool should only touch the Graal > directories in OpenJDK. If not, it's a bug and will be fixed. Sure, but it's tricky to think about how the process works. If I make back-end changes then I sometimes have to make small tweaks to the Graal sources inside OpendJDK so as not to break builds. So, I'll have to keep two parallel build trees. >> I guess I can live with that for now, but as Graal is part of >> OpenJDK and seems likely to be important we're going to have to >> make it easier for people to work on. In the long run, as you >> might expect, I'd prefer it to be a first-class part of OpenJDK. > > Yes, I agree that would simplify the OpenJDK development > process. However, Graal is used in a number of other contexts and as > such needs to be developed and tested independently of OpenJDK. We > will continue to try mitigate OpenJDK development pain by providing > tools such as updategraalinopenjdk. Fair enough. My primary concern is -- of course -- how to minimize pain for myself, but also to encourage more contributors to OpenJDK. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Wed May 2 14:37:55 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 2 May 2018 15:37:55 +0100 Subject: Can't run Graal with JDK 11 In-Reply-To: <787ADE77-4C71-4B21-A0AD-110C5ABB7858@oracle.com> References: <5649762d-38e2-f857-d36b-74ccc473366f@redhat.com> <21ea33fb-57ad-6c13-0f10-b727cdc8fa2e@redhat.com> <81c800e0-2f7f-126e-b646-6e4465d4ee5a@redhat.com> <787ADE77-4C71-4B21-A0AD-110C5ABB7858@oracle.com> Message-ID: <1e61c23a-8727-636e-bf34-0d28d5a621f2@redhat.com> On 02/05/18 10:18, Doug Simon wrote: > We will continue to try mitigate OpenJDK development pain by providing tools such as updategraalinopenjdk. Well, that worked like a charm! Thank you, it makes a big difference. From raffaello.giulietti at supsi.ch Wed May 2 17:19:14 2018 From: raffaello.giulietti at supsi.ch (raffaello.giulietti at supsi.ch) Date: Wed, 2 May 2018 19:19:14 +0200 Subject: GraalVM node: --agent option seems unknown Message-ID: <8dec32a0-387e-f455-89db-e6915060a6bd@supsi.ch> Hi, I'm not sure if this is the right list for my question below. If not, sorry for that. I'm trying to follow the GraalVM example with the Monitoring Agent http://www.graalvm.org/docs/reference-manual/tools/#monitoring-agent but cannot pass the --agent option to node: it seems unknown for some reason. I'm with the Enterprise edition of GraalVM (1.0.0-rc1) and on Linux. Yes, I checked that node is GraalVM's own node (v8.9.4) Greetings Raffaello From christian.humer at gmail.com Wed May 2 18:09:12 2018 From: christian.humer at gmail.com (Christian Humer) Date: Wed, 02 May 2018 18:09:12 +0000 Subject: GraalVM node: --agent option seems unknown In-Reply-To: <8dec32a0-387e-f455-89db-e6915060a6bd@supsi.ch> References: <8dec32a0-387e-f455-89db-e6915060a6bd@supsi.ch> Message-ID: Hi Raffaello, Thanks a lot for the report. Unfortunately in RC1 none of the tools made it into the native-image for our node executable due to some glitch in our build system. The workaround is to also pass in the --jvm option. For example: node --agent --jvm. This is fixed with RC2 that is scheduled for start of June. - Christian Humer On Wed, May 2, 2018 at 7:19 PM wrote: > Hi, > > I'm not sure if this is the right list for my question below. If not, > sorry for that. > > I'm trying to follow the GraalVM example with the Monitoring Agent > http://www.graalvm.org/docs/reference-manual/tools/#monitoring-agent > but cannot pass the --agent option to node: it seems unknown for some > reason. > > I'm with the Enterprise edition of GraalVM (1.0.0-rc1) and on Linux. > Yes, I checked that node is GraalVM's own node (v8.9.4) > > > Greetings > Raffaello > > From raffaello.giulietti at supsi.ch Wed May 2 18:27:43 2018 From: raffaello.giulietti at supsi.ch (raffaello.giulietti at supsi.ch) Date: Wed, 2 May 2018 20:27:43 +0200 Subject: GraalVM node: --agent option seems unknown In-Reply-To: References: <8dec32a0-387e-f455-89db-e6915060a6bd@supsi.ch> Message-ID: Hi Christian, indeed, I can confirm that it now works. Thanks Raffaello On 2018-05-02 20:09, Christian Humer wrote: > Hi Raffaello, > > Thanks a lot for the report. Unfortunately in RC1 none of the tools made > it into the native-image for our node executable due to some glitch in > our build system. > > The workaround is to also pass in the --jvm option. For example: node > --agent --jvm. > > This is fixed with RC2 that is scheduled for start of June. > > - Christian Humer > > > > > > On Wed, May 2, 2018 at 7:19 PM > wrote: > > Hi, > > I'm not sure if this is the right list for my question below. If not, > sorry for that. > > I'm trying to follow the GraalVM example with the Monitoring Agent > http://www.graalvm.org/docs/reference-manual/tools/#monitoring-agent > but cannot pass the --agent option to node: it seems unknown for some > reason. > > I'm with the Enterprise edition of GraalVM (1.0.0-rc1) and on Linux. > Yes, I checked that node is GraalVM's own node (v8.9.4) > > > Greetings > Raffaello > From cristian.esquivias at gmail.com Thu May 3 06:57:15 2018 From: cristian.esquivias at gmail.com (Cristian Esquivias) Date: Wed, 2 May 2018 23:57:15 -0700 Subject: OutOfMemoryError on my BF interpreter In-Reply-To: References: <6435958d-15ab-aef0-f499-96ba655e5fa0@jku.at> Message-ID: @Boris Thanks for the tip. I saw that in the code but wasn't sure why it was in there. Adding a comment to explain why @TruffleBoundary is used could be helpful for the next person. Sadly, the change didn't make a difference. I made the changes to the print [1] and read [2] nodes, but I still get the same OOME at the same spot. [1] https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/PrintDataNode.java [2] https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/ReadDataNode.java @Thomas I only have two variables stored in my function's VirtualFrame: an int and a large (1,000-10,000 length) array of bytes The README [3] explains how to build and run the interpreter. I copied simplelanguage's build process so it's basically `mvn package && JAVA_HOME=/path/to/graalvm ./bf ./examples/mandelbrot.bf` in the root directory [3] https://github.com/cesquivias/bf-graal/blob/master/README.md @Josef I'm still trying to run my interpreter off of the latest source code. I think it built fine, but I'm not sure how to connect the compiler to my Java 9 JDK. I'm using mx vm but I get an error: > mx vm -XX:+UseJVMCICompiler -Dtruffle.class.path.append=./bf-graal/lang/target/bf-1.0-SNAPSHOT.jar -cp ./bf-graal/main/target/main-1.0-SNAPSHOT.jar cesquivias.bf.Main ./bf-graal/examples/mandelbrot.bf Exception in thread "main" java.lang.IllegalStateException: A language with id 'bf' is not installed. Installed languages are: []. The compiler README [4] is a little out of date. It references the old graalvm repo though that repo says it's deprecated and to use the one under oracle org. I'm not sure if I'm calling graal right or referencing truffle properly. There wouldn't be maven repo that builds a snapshot of the graalvm that I could use instead? ;-) [4] https://github.com/oracle/graal/blob/master/compiler/README.md Thanks, Cristian On Wed, May 2, 2018 at 2:18 AM, Boris Spasojevic wrote: > Hey Christian, > > This could very well be caused by the calls to System.out.print and > System.in.read. > > Could you please try wrapping calls to such external methods and adding a > @TruffleBoundary annotation? The end goal would be something like: > > public class PrintDataNode extends BFNode { > private final FrameSlot ptr; > private final FrameSlot data; > > public PrintDataNode(FrameSlot ptr, FrameSlot data) { > this.ptr = ptr; > this.data = data; > } > > @TruffleBoundary > void print(char c) { > System.out.print(c); > } > > @Override > public void execute(VirtualFrame frame) { > try { > print((char) ((byte[]) > frame.getObject(data))[frame.getInt(ptr)]); > } catch (FrameSlotTypeException e) { > e.printStackTrace(); > } > } > } > > You can see a similar pattern being used in Simple language > (https://github.com/graalvm/simplelanguage/blob/master/language/src/main/java/com/oracle/truffle/sl/builtins/SLPrintlnBuiltin.java) > > Hope that helps! > BoriS > > > On 05/02/2018 10:46 AM, Cristian Esquivias wrote: >> >> Hi Josef, >> >> Thanks for the context. I'll try to build the GraalVM and see if it >> solves the issue. >> >> After reading the bug report, here's a little bit more info that I hope >> helps: >> >> * The report says the SSA register allocator could be the problem on >> "large inputs". I'm not sure what constitutes a large input, but my >> mandelbrot script creates around 11,000 nodes in total. That's >> probably not a huge amount, but they're all within one function since >> BF doesn't have the concept of functions. That's probably a little >> more unusual. >> * My interpreter doesn't put pressure on the heap. After an initial >> allocation of a 1,000-10,000 size int array no more allocations are >> done for the rest of a script's life. This is proven out by jvisualvm. >> The heap stays flat throughout most of the run and only spikes right >> before the OOME occurs. >> * There are very few (essentially no) opportunities for adding >> TruffleBoundary annotations since nearly all my nodes' execute methods >> are one line that do little more than fetch an item out of my array >> and inc/dec the value. The only calls out to external methods are >> calls to VirtualFrame.get [1]/set [2] some variable values, calls to >> stdout's print method [3] and stdin's read method [4]. A truffle >> expert's help would definitely be appreciated here since I'm just >> making an educated guess. >> * The OOME occurs inside a loop node. Could this OOME be caused by >> loop unrolling [5]? >> >> Thanks, >> Cristian >> >> [1]: >> https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/DecDataNode.java >> [2]: >> https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/DecPointerNode.java >> [3]: >> https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/PrintDataNode.java >> [4]: >> https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/ReadDataNode.java >> [5]: >> https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/BFRepeatingNode.java >> >> On Wed, May 2, 2018 at 12:37 AM, Josef Eisl wrote: >>> >>> Hi Cristian! >>> >>> Thanks for your report. We have very recently merged a change to >>> mitigate the OOME in this part of linear scan [1]. Unfortunately this >>> was after the release so it is not included in 1.0.0 RC1. You would need >>> to build your own GraalVM from source to see if the change helps. >>> >>> That said, the root cause of the OOME is usually a huge compilation unit >>> thrown into the compiler. For more details, see also the discussion on >>> the OpenJDK bug tracker [2]. >>> The reasons for big compilation units can me manifold. In case of >>> Truffle, the issues can often be solved by adding TruffleBoundaries [3], >>> i.e., instructing the compiler not to inline specified methods into the >>> truffle graph. Maybe a Truffle expert can give you more hints on that. >>> >>> Hope that helps, >>> >>> Josef >>> >>> [1]: >>> >>> https://github.com/oracle/graal/commit/c0a15c562daa0338f61a6e7c22476cf145af5a66 >>> [2]: https://bugs.openjdk.java.net/browse/JDK-8199890 >>> [3]: >>> >>> http://www.graalvm.org/truffle/javadoc/com/oracle/truffle/api/CompilerDirectives.TruffleBoundary.html >>> >>> On 01/05/18 08:24, Cristian Esquivias wrote: >>>> >>>> With the release of 1.0.0-RC1 I thought I'd come back and see how >>>> Graal is doing. Great job; Graal is really starting to look like the >>>> platform everyone should pay attention to. >>>> >>>> To get back into it, I built a Brainf*ck interpreter with Truffle >>>> (https://github.com/cesquivias/bf-graal). I modeled it after >>>> simplelanguage so it should be easy to build and run. >>>> >>>> While running the mandelbrot program, about halfway through the >>>> program, the VM starts throwing OutOfMemoryError exceptions (one >>>> pasted below). It created a zip file under dumps/ but it contains a >>>> .bgv file I don't what to do with and a log file that just contains >>>> the stacktrace. >>>> >>>> I ran jvisualvm and took a heap dump. There are ~200k long[] objects >>>> alone created by graal taking up ~50% of the memory. Digging through >>>> the references it seems to be created/retained in >>>> >>>> >>>> org.graalvm.compiler.lir.alloc.lsra.LinearScanLifetimeAnalysisPhase$$Lambda$48#1 >>>> [GC root - Java frame] >>>> >>>> I'd provide some file from jvisualvm but the save button is grayed >>>> out. I hope this is enough info. My interpreter is up on GitHub so >>>> feel free to test it out. >>>> >>>> I tested this both on the community & enterprise edition on Ubuntu >>>> 18.04 in VirtualBox. >>>> >>>> - Cristian >>>> >>>> >>>> Thread[TruffleCompilerThread-12,10,main]: Compilation of >>>> BFRepeatingNode at 222b298d failed: java.lang.OutOfMemoryError: Java >>>> heap space >>>> at >>>> org.graalvm.compiler.lir.util.IndexedValueMap.(IndexedValueMap.java:55) >>>> at >>>> org.graalvm.compiler.lir.dfa.RegStackValueSet.(RegStackValueSet.java:62) >>>> at >>>> org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:70) >>>> at >>>> org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:46) >>>> at >>>> org.graalvm.compiler.lir.dfa.LocationMarker.processBlock(LocationMarker.java:107) >>>> at >>>> org.graalvm.compiler.lir.dfa.LocationMarker.build(LocationMarker.java:81) >>>> at >>>> org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:51) >>>> at >>>> org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:47) >>>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) >>>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) >>>> at >>>> org.graalvm.compiler.lir.phases.LIRPhaseSuite.run(LIRPhaseSuite.java:96) >>>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) >>>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) >>>> at >>>> org.graalvm.compiler.core.GraalCompiler.emitLowLevel(GraalCompiler.java:367) >>>> at >>>> org.graalvm.compiler.core.GraalCompiler.emitLIR0(GraalCompiler.java:336) >>>> at >>>> org.graalvm.compiler.core.GraalCompiler.emitLIR(GraalCompiler.java:295) >>>> at >>>> org.graalvm.compiler.core.GraalCompiler.emitBackEnd(GraalCompiler.java:275) >>>> at >>>> org.graalvm.compiler.core.GraalCompiler.compile(GraalCompiler.java:175) >>>> at >>>> org.graalvm.compiler.core.GraalCompiler.compileGraph(GraalCompiler.java:160) >>>> at >>>> org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compilePEGraph(TruffleCompilerImpl.java:445) >>>> at >>>> org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compileAST(TruffleCompilerImpl.java:391) >>>> at >>>> org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:544) >>>> at >>>> org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:493) >>>> at >>>> org.graalvm.compiler.core.CompilationWrapper.run(CompilationWrapper.java:167) >>>> at >>>> org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.doCompile(TruffleCompilerImpl.java:222) >>>> at >>>> org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime.doCompile(GraalTruffleRuntime.java:679) >>>> at >>>> org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime$1.run(GraalTruffleRuntime.java:745) >>>> at >>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >>>> at java.lang.Thread.run(Thread.java:748) >>>> To disable compilation failure notifications, set >>>> CompilationFailureAction to Silent (e.g., >>>> -Dgraal.CompilationFailureAction=Silent). >>>> To print a message for a compilation failure without retrying the >>>> compilation, set CompilationFailureAction to Print (e.g., >>>> -Dgraal.CompilationFailureAction=Print). >>>> Retrying compilation of BFRepeatingNode at 222b298d >>>> > From boris.spasojevic at oracle.com Thu May 3 08:01:46 2018 From: boris.spasojevic at oracle.com (Boris Spasojevic) Date: Thu, 3 May 2018 10:01:46 +0200 Subject: OutOfMemoryError on my BF interpreter In-Reply-To: References: <6435958d-15ab-aef0-f499-96ba655e5fa0@jku.at> Message-ID: Hey Cristian, Yeah, placing Truffle boundaries is a bit counterintuitive if one doesn't actively think about the partial evaluation. A simple rule of thumb basically is to have a TruffleBoundary when calling any code that's out of your control (calls to library code for example, standard library included). This is not guaranteed to be sufficient , but it caches a lot of problems caused by a missing TruffleBoundary. With that said, I would also recommend a TruffleBoundary before calling printStackTrace as it is basically a System.err.println Cheers, BoriS On 05/03/2018 08:57 AM, Cristian Esquivias wrote: > @Boris > Thanks for the tip. I saw that in the code but wasn't sure why it was > in there. Adding a comment to explain why @TruffleBoundary is used > could be helpful for the next person. > > Sadly, the change didn't make a difference. I made the changes to the > print [1] and read [2] nodes, but I still get the same OOME at the > same spot. > > [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cesquivias_bf-2Dgraal_blob_master_lang_src_main_java_cesquivias_bf_PrintDataNode.java&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=NwNmAv3qE59ZjkeHn5Yud4_C4XfMw6Xa7kM8PVY7CGA&m=dXO26nNo7uW2-OUngRCOYOjy8XsDAg1nOz-al_kjiDo&s=XHQ0a5p5oZndQdhC47G43J5sK6yqBL2B6quxhECa9kc&e= > [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cesquivias_bf-2Dgraal_blob_master_lang_src_main_java_cesquivias_bf_ReadDataNode.java&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=NwNmAv3qE59ZjkeHn5Yud4_C4XfMw6Xa7kM8PVY7CGA&m=dXO26nNo7uW2-OUngRCOYOjy8XsDAg1nOz-al_kjiDo&s=bNMbb0YzOBTpiswbkS4kWPcHndxUBbJ4nfN744tn-_A&e= > > > @Thomas > I only have two variables stored in my function's VirtualFrame: an int > and a large (1,000-10,000 length) array of bytes > > The README [3] explains how to build and run the interpreter. I copied > simplelanguage's build process so it's basically `mvn package && > JAVA_HOME=/path/to/graalvm ./bf ./examples/mandelbrot.bf` in the root > directory > > [3] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cesquivias_bf-2Dgraal_blob_master_README.md&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=NwNmAv3qE59ZjkeHn5Yud4_C4XfMw6Xa7kM8PVY7CGA&m=dXO26nNo7uW2-OUngRCOYOjy8XsDAg1nOz-al_kjiDo&s=-1-AzVo8qqKnlaLPCon2ASMq37E_P3WMnIHmhdR4BsY&e= > > > @Josef > I'm still trying to run my interpreter off of the latest source code. > I think it built fine, but I'm not sure how to connect the compiler to > my Java 9 JDK. I'm using mx vm but I get an error: > >> mx vm -XX:+UseJVMCICompiler -Dtruffle.class.path.append=./bf-graal/lang/target/bf-1.0-SNAPSHOT.jar -cp ./bf-graal/main/target/main-1.0-SNAPSHOT.jar cesquivias.bf.Main ./bf-graal/examples/mandelbrot.bf > Exception in thread "main" java.lang.IllegalStateException: A language > with id 'bf' is not installed. Installed languages are: []. > > The compiler README [4] is a little out of date. It references the old > graalvm repo though that repo says it's deprecated and to use the one > under oracle org. I'm not sure if I'm calling graal right or > referencing truffle properly. There wouldn't be maven repo that builds > a snapshot of the graalvm that I could use instead? ;-) > > [4] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_oracle_graal_blob_master_compiler_README.md&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=NwNmAv3qE59ZjkeHn5Yud4_C4XfMw6Xa7kM8PVY7CGA&m=dXO26nNo7uW2-OUngRCOYOjy8XsDAg1nOz-al_kjiDo&s=2a3Z0BOiZ3BcnNk7Pf1KvavZFMpll7WlQC2ZOVb21dc&e= > > > Thanks, > Cristian > > On Wed, May 2, 2018 at 2:18 AM, Boris Spasojevic > wrote: >> Hey Christian, >> >> This could very well be caused by the calls to System.out.print and >> System.in.read. >> >> Could you please try wrapping calls to such external methods and adding a >> @TruffleBoundary annotation? The end goal would be something like: >> >> public class PrintDataNode extends BFNode { >> private final FrameSlot ptr; >> private final FrameSlot data; >> >> public PrintDataNode(FrameSlot ptr, FrameSlot data) { >> this.ptr = ptr; >> this.data = data; >> } >> >> @TruffleBoundary >> void print(char c) { >> System.out.print(c); >> } >> >> @Override >> public void execute(VirtualFrame frame) { >> try { >> print((char) ((byte[]) >> frame.getObject(data))[frame.getInt(ptr)]); >> } catch (FrameSlotTypeException e) { >> e.printStackTrace(); >> } >> } >> } >> >> You can see a similar pattern being used in Simple language >> (https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_graalvm_simplelanguage_blob_master_language_src_main_java_com_oracle_truffle_sl_builtins_SLPrintlnBuiltin.java&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=NwNmAv3qE59ZjkeHn5Yud4_C4XfMw6Xa7kM8PVY7CGA&m=dXO26nNo7uW2-OUngRCOYOjy8XsDAg1nOz-al_kjiDo&s=giidu6TjqemIVn_pLYhakLSzU-TkIyYEdixHr2NUm7s&e=) >> >> Hope that helps! >> BoriS >> >> >> On 05/02/2018 10:46 AM, Cristian Esquivias wrote: >>> Hi Josef, >>> >>> Thanks for the context. I'll try to build the GraalVM and see if it >>> solves the issue. >>> >>> After reading the bug report, here's a little bit more info that I hope >>> helps: >>> >>> * The report says the SSA register allocator could be the problem on >>> "large inputs". I'm not sure what constitutes a large input, but my >>> mandelbrot script creates around 11,000 nodes in total. That's >>> probably not a huge amount, but they're all within one function since >>> BF doesn't have the concept of functions. That's probably a little >>> more unusual. >>> * My interpreter doesn't put pressure on the heap. After an initial >>> allocation of a 1,000-10,000 size int array no more allocations are >>> done for the rest of a script's life. This is proven out by jvisualvm. >>> The heap stays flat throughout most of the run and only spikes right >>> before the OOME occurs. >>> * There are very few (essentially no) opportunities for adding >>> TruffleBoundary annotations since nearly all my nodes' execute methods >>> are one line that do little more than fetch an item out of my array >>> and inc/dec the value. The only calls out to external methods are >>> calls to VirtualFrame.get [1]/set [2] some variable values, calls to >>> stdout's print method [3] and stdin's read method [4]. A truffle >>> expert's help would definitely be appreciated here since I'm just >>> making an educated guess. >>> * The OOME occurs inside a loop node. Could this OOME be caused by >>> loop unrolling [5]? >>> >>> Thanks, >>> Cristian >>> >>> [1]: >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cesquivias_bf-2Dgraal_blob_master_lang_src_main_java_cesquivias_bf_DecDataNode.java&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=NwNmAv3qE59ZjkeHn5Yud4_C4XfMw6Xa7kM8PVY7CGA&m=dXO26nNo7uW2-OUngRCOYOjy8XsDAg1nOz-al_kjiDo&s=IVqgl6m3CiYHQee_fG23LJBm-NVM4n6dXjFd6Kfh9KE&e= >>> [2]: >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cesquivias_bf-2Dgraal_blob_master_lang_src_main_java_cesquivias_bf_DecPointerNode.java&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=NwNmAv3qE59ZjkeHn5Yud4_C4XfMw6Xa7kM8PVY7CGA&m=dXO26nNo7uW2-OUngRCOYOjy8XsDAg1nOz-al_kjiDo&s=-TXnuUXPvxuHGrnR3G2_VUat55zVhcISswe2sGtrGJE&e= >>> [3]: >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cesquivias_bf-2Dgraal_blob_master_lang_src_main_java_cesquivias_bf_PrintDataNode.java&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=NwNmAv3qE59ZjkeHn5Yud4_C4XfMw6Xa7kM8PVY7CGA&m=dXO26nNo7uW2-OUngRCOYOjy8XsDAg1nOz-al_kjiDo&s=XHQ0a5p5oZndQdhC47G43J5sK6yqBL2B6quxhECa9kc&e= >>> [4]: >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cesquivias_bf-2Dgraal_blob_master_lang_src_main_java_cesquivias_bf_ReadDataNode.java&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=NwNmAv3qE59ZjkeHn5Yud4_C4XfMw6Xa7kM8PVY7CGA&m=dXO26nNo7uW2-OUngRCOYOjy8XsDAg1nOz-al_kjiDo&s=bNMbb0YzOBTpiswbkS4kWPcHndxUBbJ4nfN744tn-_A&e= >>> [5]: >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cesquivias_bf-2Dgraal_blob_master_lang_src_main_java_cesquivias_bf_BFRepeatingNode.java&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=NwNmAv3qE59ZjkeHn5Yud4_C4XfMw6Xa7kM8PVY7CGA&m=dXO26nNo7uW2-OUngRCOYOjy8XsDAg1nOz-al_kjiDo&s=3bfEd_51FCE6DFloTP4rkquO0kSaeFtVFwCS658QUak&e= >>> >>> On Wed, May 2, 2018 at 12:37 AM, Josef Eisl wrote: >>>> Hi Cristian! >>>> >>>> Thanks for your report. We have very recently merged a change to >>>> mitigate the OOME in this part of linear scan [1]. Unfortunately this >>>> was after the release so it is not included in 1.0.0 RC1. You would need >>>> to build your own GraalVM from source to see if the change helps. >>>> >>>> That said, the root cause of the OOME is usually a huge compilation unit >>>> thrown into the compiler. For more details, see also the discussion on >>>> the OpenJDK bug tracker [2]. >>>> The reasons for big compilation units can me manifold. In case of >>>> Truffle, the issues can often be solved by adding TruffleBoundaries [3], >>>> i.e., instructing the compiler not to inline specified methods into the >>>> truffle graph. Maybe a Truffle expert can give you more hints on that. >>>> >>>> Hope that helps, >>>> >>>> Josef >>>> >>>> [1]: >>>> >>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_oracle_graal_commit_c0a15c562daa0338f61a6e7c22476cf145af5a66&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=NwNmAv3qE59ZjkeHn5Yud4_C4XfMw6Xa7kM8PVY7CGA&m=dXO26nNo7uW2-OUngRCOYOjy8XsDAg1nOz-al_kjiDo&s=7YLDLYqar4_8tMJzlvNYCt8hFzQNdAH2MgLKpEl-HGo&e= >>>> [2]: https://bugs.openjdk.java.net/browse/JDK-8199890 >>>> [3]: >>>> >>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.graalvm.org_truffle_javadoc_com_oracle_truffle_api_CompilerDirectives.TruffleBoundary.html&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=NwNmAv3qE59ZjkeHn5Yud4_C4XfMw6Xa7kM8PVY7CGA&m=dXO26nNo7uW2-OUngRCOYOjy8XsDAg1nOz-al_kjiDo&s=2hdW0qEgkD7u2vIp8yWOQYf3jx6DVSkesQGUEoEfazE&e= >>>> >>>> On 01/05/18 08:24, Cristian Esquivias wrote: >>>>> With the release of 1.0.0-RC1 I thought I'd come back and see how >>>>> Graal is doing. Great job; Graal is really starting to look like the >>>>> platform everyone should pay attention to. >>>>> >>>>> To get back into it, I built a Brainf*ck interpreter with Truffle >>>>> (https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cesquivias_bf-2Dgraal&d=DwIBaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=NwNmAv3qE59ZjkeHn5Yud4_C4XfMw6Xa7kM8PVY7CGA&m=dXO26nNo7uW2-OUngRCOYOjy8XsDAg1nOz-al_kjiDo&s=tqWPZH7230x0xGzs_O8_vIBObFrb8-1F9iSrUdTZdv4&e=). I modeled it after >>>>> simplelanguage so it should be easy to build and run. >>>>> >>>>> While running the mandelbrot program, about halfway through the >>>>> program, the VM starts throwing OutOfMemoryError exceptions (one >>>>> pasted below). It created a zip file under dumps/ but it contains a >>>>> .bgv file I don't what to do with and a log file that just contains >>>>> the stacktrace. >>>>> >>>>> I ran jvisualvm and took a heap dump. There are ~200k long[] objects >>>>> alone created by graal taking up ~50% of the memory. Digging through >>>>> the references it seems to be created/retained in >>>>> >>>>> >>>>> org.graalvm.compiler.lir.alloc.lsra.LinearScanLifetimeAnalysisPhase$$Lambda$48#1 >>>>> [GC root - Java frame] >>>>> >>>>> I'd provide some file from jvisualvm but the save button is grayed >>>>> out. I hope this is enough info. My interpreter is up on GitHub so >>>>> feel free to test it out. >>>>> >>>>> I tested this both on the community & enterprise edition on Ubuntu >>>>> 18.04 in VirtualBox. >>>>> >>>>> - Cristian >>>>> >>>>> >>>>> Thread[TruffleCompilerThread-12,10,main]: Compilation of >>>>> BFRepeatingNode at 222b298d failed: java.lang.OutOfMemoryError: Java >>>>> heap space >>>>> at >>>>> org.graalvm.compiler.lir.util.IndexedValueMap.(IndexedValueMap.java:55) >>>>> at >>>>> org.graalvm.compiler.lir.dfa.RegStackValueSet.(RegStackValueSet.java:62) >>>>> at >>>>> org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:70) >>>>> at >>>>> org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:46) >>>>> at >>>>> org.graalvm.compiler.lir.dfa.LocationMarker.processBlock(LocationMarker.java:107) >>>>> at >>>>> org.graalvm.compiler.lir.dfa.LocationMarker.build(LocationMarker.java:81) >>>>> at >>>>> org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:51) >>>>> at >>>>> org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:47) >>>>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) >>>>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) >>>>> at >>>>> org.graalvm.compiler.lir.phases.LIRPhaseSuite.run(LIRPhaseSuite.java:96) >>>>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) >>>>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) >>>>> at >>>>> org.graalvm.compiler.core.GraalCompiler.emitLowLevel(GraalCompiler.java:367) >>>>> at >>>>> org.graalvm.compiler.core.GraalCompiler.emitLIR0(GraalCompiler.java:336) >>>>> at >>>>> org.graalvm.compiler.core.GraalCompiler.emitLIR(GraalCompiler.java:295) >>>>> at >>>>> org.graalvm.compiler.core.GraalCompiler.emitBackEnd(GraalCompiler.java:275) >>>>> at >>>>> org.graalvm.compiler.core.GraalCompiler.compile(GraalCompiler.java:175) >>>>> at >>>>> org.graalvm.compiler.core.GraalCompiler.compileGraph(GraalCompiler.java:160) >>>>> at >>>>> org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compilePEGraph(TruffleCompilerImpl.java:445) >>>>> at >>>>> org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compileAST(TruffleCompilerImpl.java:391) >>>>> at >>>>> org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:544) >>>>> at >>>>> org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:493) >>>>> at >>>>> org.graalvm.compiler.core.CompilationWrapper.run(CompilationWrapper.java:167) >>>>> at >>>>> org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.doCompile(TruffleCompilerImpl.java:222) >>>>> at >>>>> org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime.doCompile(GraalTruffleRuntime.java:679) >>>>> at >>>>> org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime$1.run(GraalTruffleRuntime.java:745) >>>>> at >>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >>>>> at java.lang.Thread.run(Thread.java:748) >>>>> To disable compilation failure notifications, set >>>>> CompilationFailureAction to Silent (e.g., >>>>> -Dgraal.CompilationFailureAction=Silent). >>>>> To print a message for a compilation failure without retrying the >>>>> compilation, set CompilationFailureAction to Print (e.g., >>>>> -Dgraal.CompilationFailureAction=Print). >>>>> Retrying compilation of BFRepeatingNode at 222b298d >>>>> From christian.humer at gmail.com Fri May 4 10:50:20 2018 From: christian.humer at gmail.com (Christian Humer) Date: Fri, 04 May 2018 10:50:20 +0000 Subject: OutOfMemoryError on my BF interpreter In-Reply-To: References: <6435958d-15ab-aef0-f499-96ba655e5fa0@jku.at> Message-ID: Hi Cristian, This command line should pick up the latest compiler: mx -v --jdk=jvmci vm -cp ../../bf-graal/main/target/main-1.0-SNAPSHOT.jar:../../bf-graal/lang/target/bf-1.0-SNAPSHOT.jar cesquivias.bf.Main ../../bf-graal/examples/mandelbrot.bf I ran this command out of the graal compiler suite in graal/compiler folder of the Graal repository. I checked out the bf repo at the root, hence the ../../. Note that the -Dtruffle.class.path.append is only a thing in GraalVM so you need to use -cp when running with the Graal repository. The -v command gives you some info what command it produces. Feel free to remove it. We are working on deploying nightly snapshots of GraalVM. That should help you picking up the latest version in the future. I took the liberty to have a look at a more reasonable BF application here: https://github.com/chumer/bf/blob/master/src/main/resources/test/primes.bf I was dumping this to IGV (mx igv in compiler project) and then running using -Dgraal.Dump=. This showed me a few problems with the graph. You should have a look at the "nodeSourcePosition" property that shows you the origin of Graal nodes. I created a PR with the changes that should fix those problems: https://github.com/cesquivias/bf-graal/pull/1 When I look at the IGV graph now I can see only loads and stores on your memory int[]. Unfortunately Graal needs to materialize all the changes in the memory as it cannot see its allocation in the OSR compilations. You could try to materialize every memory slot into a frame slot. But I'd rather recommend spending your time on a more interesting language like a simple Lisp. BTW.: I did my own BF interpreter two years ago: https://github.com/chumer/bf/. Looks very similar to yours :-). Hope this helps, - Christian Humer On Thu, May 3, 2018 at 8:58 AM Cristian Esquivias < cristian.esquivias at gmail.com> wrote: > @Boris > Thanks for the tip. I saw that in the code but wasn't sure why it was > in there. Adding a comment to explain why @TruffleBoundary is used > could be helpful for the next person. > > Sadly, the change didn't make a difference. I made the changes to the > print [1] and read [2] nodes, but I still get the same OOME at the > same spot. > > [1] > https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/PrintDataNode.java > [2] > https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/ReadDataNode.java > > > @Thomas > I only have two variables stored in my function's VirtualFrame: an int > and a large (1,000-10,000 length) array of bytes > > The README [3] explains how to build and run the interpreter. I copied > simplelanguage's build process so it's basically `mvn package && > JAVA_HOME=/path/to/graalvm ./bf ./examples/mandelbrot.bf` in the root > directory > > [3] https://github.com/cesquivias/bf-graal/blob/master/README.md > > > @Josef > I'm still trying to run my interpreter off of the latest source code. > I think it built fine, but I'm not sure how to connect the compiler to > my Java 9 JDK. I'm using mx vm but I get an error: > > > mx vm -XX:+UseJVMCICompiler > -Dtruffle.class.path.append=./bf-graal/lang/target/bf-1.0-SNAPSHOT.jar -cp > ./bf-graal/main/target/main-1.0-SNAPSHOT.jar cesquivias.bf.Main > ./bf-graal/examples/mandelbrot.bf > Exception in thread "main" java.lang.IllegalStateException: A language > with id 'bf' is not installed. Installed languages are: []. > > The compiler README [4] is a little out of date. It references the old > graalvm repo though that repo says it's deprecated and to use the one > under oracle org. I'm not sure if I'm calling graal right or > referencing truffle properly. There wouldn't be maven repo that builds > a snapshot of the graalvm that I could use instead? ;-) > > [4] https://github.com/oracle/graal/blob/master/compiler/README.md > > > Thanks, > Cristian > > On Wed, May 2, 2018 at 2:18 AM, Boris Spasojevic > wrote: > > Hey Christian, > > > > This could very well be caused by the calls to System.out.print and > > System.in.read. > > > > Could you please try wrapping calls to such external methods and adding a > > @TruffleBoundary annotation? The end goal would be something like: > > > > public class PrintDataNode extends BFNode { > > private final FrameSlot ptr; > > private final FrameSlot data; > > > > public PrintDataNode(FrameSlot ptr, FrameSlot data) { > > this.ptr = ptr; > > this.data = data; > > } > > > > @TruffleBoundary > > void print(char c) { > > System.out.print(c); > > } > > > > @Override > > public void execute(VirtualFrame frame) { > > try { > > print((char) ((byte[]) > > frame.getObject(data))[frame.getInt(ptr)]); > > } catch (FrameSlotTypeException e) { > > e.printStackTrace(); > > } > > } > > } > > > > You can see a similar pattern being used in Simple language > > ( > https://github.com/graalvm/simplelanguage/blob/master/language/src/main/java/com/oracle/truffle/sl/builtins/SLPrintlnBuiltin.java > ) > > > > Hope that helps! > > BoriS > > > > > > On 05/02/2018 10:46 AM, Cristian Esquivias wrote: > >> > >> Hi Josef, > >> > >> Thanks for the context. I'll try to build the GraalVM and see if it > >> solves the issue. > >> > >> After reading the bug report, here's a little bit more info that I hope > >> helps: > >> > >> * The report says the SSA register allocator could be the problem on > >> "large inputs". I'm not sure what constitutes a large input, but my > >> mandelbrot script creates around 11,000 nodes in total. That's > >> probably not a huge amount, but they're all within one function since > >> BF doesn't have the concept of functions. That's probably a little > >> more unusual. > >> * My interpreter doesn't put pressure on the heap. After an initial > >> allocation of a 1,000-10,000 size int array no more allocations are > >> done for the rest of a script's life. This is proven out by jvisualvm. > >> The heap stays flat throughout most of the run and only spikes right > >> before the OOME occurs. > >> * There are very few (essentially no) opportunities for adding > >> TruffleBoundary annotations since nearly all my nodes' execute methods > >> are one line that do little more than fetch an item out of my array > >> and inc/dec the value. The only calls out to external methods are > >> calls to VirtualFrame.get [1]/set [2] some variable values, calls to > >> stdout's print method [3] and stdin's read method [4]. A truffle > >> expert's help would definitely be appreciated here since I'm just > >> making an educated guess. > >> * The OOME occurs inside a loop node. Could this OOME be caused by > >> loop unrolling [5]? > >> > >> Thanks, > >> Cristian > >> > >> [1]: > >> > https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/DecDataNode.java > >> [2]: > >> > https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/DecPointerNode.java > >> [3]: > >> > https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/PrintDataNode.java > >> [4]: > >> > https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/ReadDataNode.java > >> [5]: > >> > https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/BFRepeatingNode.java > >> > >> On Wed, May 2, 2018 at 12:37 AM, Josef Eisl wrote: > >>> > >>> Hi Cristian! > >>> > >>> Thanks for your report. We have very recently merged a change to > >>> mitigate the OOME in this part of linear scan [1]. Unfortunately this > >>> was after the release so it is not included in 1.0.0 RC1. You would > need > >>> to build your own GraalVM from source to see if the change helps. > >>> > >>> That said, the root cause of the OOME is usually a huge compilation > unit > >>> thrown into the compiler. For more details, see also the discussion on > >>> the OpenJDK bug tracker [2]. > >>> The reasons for big compilation units can me manifold. In case of > >>> Truffle, the issues can often be solved by adding TruffleBoundaries > [3], > >>> i.e., instructing the compiler not to inline specified methods into the > >>> truffle graph. Maybe a Truffle expert can give you more hints on that. > >>> > >>> Hope that helps, > >>> > >>> Josef > >>> > >>> [1]: > >>> > >>> > https://github.com/oracle/graal/commit/c0a15c562daa0338f61a6e7c22476cf145af5a66 > >>> [2]: https://bugs.openjdk.java.net/browse/JDK-8199890 > >>> [3]: > >>> > >>> > http://www.graalvm.org/truffle/javadoc/com/oracle/truffle/api/CompilerDirectives.TruffleBoundary.html > >>> > >>> On 01/05/18 08:24, Cristian Esquivias wrote: > >>>> > >>>> With the release of 1.0.0-RC1 I thought I'd come back and see how > >>>> Graal is doing. Great job; Graal is really starting to look like the > >>>> platform everyone should pay attention to. > >>>> > >>>> To get back into it, I built a Brainf*ck interpreter with Truffle > >>>> (https://github.com/cesquivias/bf-graal). I modeled it after > >>>> simplelanguage so it should be easy to build and run. > >>>> > >>>> While running the mandelbrot program, about halfway through the > >>>> program, the VM starts throwing OutOfMemoryError exceptions (one > >>>> pasted below). It created a zip file under dumps/ but it contains a > >>>> .bgv file I don't what to do with and a log file that just contains > >>>> the stacktrace. > >>>> > >>>> I ran jvisualvm and took a heap dump. There are ~200k long[] objects > >>>> alone created by graal taking up ~50% of the memory. Digging through > >>>> the references it seems to be created/retained in > >>>> > >>>> > >>>> > org.graalvm.compiler.lir.alloc.lsra.LinearScanLifetimeAnalysisPhase$$Lambda$48#1 > >>>> [GC root - Java frame] > >>>> > >>>> I'd provide some file from jvisualvm but the save button is grayed > >>>> out. I hope this is enough info. My interpreter is up on GitHub so > >>>> feel free to test it out. > >>>> > >>>> I tested this both on the community & enterprise edition on Ubuntu > >>>> 18.04 in VirtualBox. > >>>> > >>>> - Cristian > >>>> > >>>> > >>>> Thread[TruffleCompilerThread-12,10,main]: Compilation of > >>>> BFRepeatingNode at 222b298d failed: java.lang.OutOfMemoryError: > Java > >>>> heap space > >>>> at > >>>> > org.graalvm.compiler.lir.util.IndexedValueMap.(IndexedValueMap.java:55) > >>>> at > >>>> > org.graalvm.compiler.lir.dfa.RegStackValueSet.(RegStackValueSet.java:62) > >>>> at > >>>> > org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:70) > >>>> at > >>>> > org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:46) > >>>> at > >>>> > org.graalvm.compiler.lir.dfa.LocationMarker.processBlock(LocationMarker.java:107) > >>>> at > >>>> > org.graalvm.compiler.lir.dfa.LocationMarker.build(LocationMarker.java:81) > >>>> at > >>>> > org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:51) > >>>> at > >>>> > org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:47) > >>>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) > >>>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) > >>>> at > >>>> > org.graalvm.compiler.lir.phases.LIRPhaseSuite.run(LIRPhaseSuite.java:96) > >>>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) > >>>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) > >>>> at > >>>> > org.graalvm.compiler.core.GraalCompiler.emitLowLevel(GraalCompiler.java:367) > >>>> at > >>>> > org.graalvm.compiler.core.GraalCompiler.emitLIR0(GraalCompiler.java:336) > >>>> at > >>>> > org.graalvm.compiler.core.GraalCompiler.emitLIR(GraalCompiler.java:295) > >>>> at > >>>> > org.graalvm.compiler.core.GraalCompiler.emitBackEnd(GraalCompiler.java:275) > >>>> at > >>>> > org.graalvm.compiler.core.GraalCompiler.compile(GraalCompiler.java:175) > >>>> at > >>>> > org.graalvm.compiler.core.GraalCompiler.compileGraph(GraalCompiler.java:160) > >>>> at > >>>> > org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compilePEGraph(TruffleCompilerImpl.java:445) > >>>> at > >>>> > org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compileAST(TruffleCompilerImpl.java:391) > >>>> at > >>>> > org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:544) > >>>> at > >>>> > org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:493) > >>>> at > >>>> > org.graalvm.compiler.core.CompilationWrapper.run(CompilationWrapper.java:167) > >>>> at > >>>> > org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.doCompile(TruffleCompilerImpl.java:222) > >>>> at > >>>> > org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime.doCompile(GraalTruffleRuntime.java:679) > >>>> at > >>>> > org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime$1.run(GraalTruffleRuntime.java:745) > >>>> at > >>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) > >>>> at > >>>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > >>>> at > >>>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > >>>> at java.lang.Thread.run(Thread.java:748) > >>>> To disable compilation failure notifications, set > >>>> CompilationFailureAction to Silent (e.g., > >>>> -Dgraal.CompilationFailureAction=Silent). > >>>> To print a message for a compilation failure without retrying the > >>>> compilation, set CompilationFailureAction to Print (e.g., > >>>> -Dgraal.CompilationFailureAction=Print). > >>>> Retrying compilation of BFRepeatingNode at 222b298d > >>>> > > > From lists at fniephaus.com Fri May 4 13:39:43 2018 From: lists at fniephaus.com (Fabio Niephaus) Date: Fri, 04 May 2018 13:39:43 +0000 Subject: 2019: Call for Papers & Call for Workshop Proposals Message-ID: 2019 : The Art, Science, and Engineering of Programming April 1-4, 2019, Genova, Italy http://2019.programming-conference.org The International Conference on the Art, Science, and Engineering of Programming is a new conference focused on programming topics including the experience of programming. We have named it for short. seeks for papers that advance knowledge of programming on any relevant topic, including programming practice and experience. Paper submissions and publications are handled by the the Art, Science, and Engineering of Programming journal (http://programming-journal.org). Accepted papers must be presented at the conference. ******************************************************** CALL FOR PAPERS ******************************************************** 2019 accepts scholarly papers that advance knowledge of programming. Almost anything about programming is in scope, but in each case there should be a clear relevance to the act and experience of programming. PAPER SUBMISSIONS: June 1, 2018 URL FOR SUBMISSIONS: http://programming-journal.org/submission/ Submissions covering several areas of expertise are accepted, including but not limited to: ? General-purpose programming ? Distributed systems programming ? Parallel and multi-core programming ? Graphics and GPU programming ? Security programming ? User interface programming ? Database programming ? Visual and live programming ? Data mining and machine learning programming ? Interpreters, virtual machines and compilers ? Modularity and separation of concerns ? Model-based development ? Metaprogramming and reflection ? Testing and debugging ? Program verification ? Programming education ? Programming environments ? Social coding ******************************************************** CALL FOR WORKSHOP PROPOSALS ******************************************************** To build a community and to foster an environment where participants can exchange ideas and experiences related to practical software development, will host a number of workshops, during the days before the main conference. The workshops will provide a collaborative forum to exchange recent and/or preliminary results, to conduct intensive discussions on a particular topic, or to coordinate efforts between representatives of a technical community. They are intended as a forum for lively discussion of innovative ideas, recent progress, or practical experience on programming and applied software development in general for specific aspects, specific problems, or domain-specific needs. We also encourage practical, hands-on workshops in which participants actually experience one or several aspects of practical software development. WORKSHOP PROPOSAL SUBMISSIONS: First Deadline: July 1st, 2018 Second Deadline: September 1st, 2018 The duration of workshops is in general one day, but we encourage the submission of half-day workshop proposals on focused topics as well. In exceptional situations, e.g., for workshops that involve actual practice of programming-related activities, workshop organizers can request a 2 day workshop slot. If desired, the workshop proceedings can be published in the ACM Digital Library. ******************************************************** IMPORTANT DATES ******************************************************** Research paper submissions: June 1, 2019 Research paper first notifications: August 1, 2018 Research paper final notifications: September 7, 2018 Workshop proposals: July 1st, 2018 (first deadline) Workshop proposals: September 1st, 2018 (second deadline) ******************************************************** ORGANIZATION ******************************************************** General Chair: Davide Ancona, University of Genova Local Organizing Chair: Elena Zucca, University of Genova Program Chair: Matthew Flatt, University of Utah Organizing Committee: Walter Cazzola (Workshops Co-Chair), Universit? degli Studi di Milano Stefan Marr (Workshops Co-Chair), University of Kent Fabio Niephaus (Publicity Co-Chair), Hasso Plattner Institute, University of Potsdam Tobias Pape (Web Technology Chair), Hasso Plattner Institute, University of Potsdam Program Committee: Mehdi Bagherzadeh, Oakland University Walter Cazzola, Universit? degli Studi di Milano Ravi Chugh, University of Chicago Joeri De Koster, Vrije Universiteit Brussel Christos Dimoulas, Northwestern University Susan Eisenbach, Imperial College London Richard P. Gabriel, Dream Songs, Inc. & HPI Jeremy Gibbons, University of Oxford Michael Greenberg, Pomona College Philipp Haller, KTH Royal Institute of Technology Robert Hirschfeld, HPI, University of Potsdam Eunsuk Kang, Carnegie Mellon University Stephen Kell, University of Cambridge Stefan Marr, University of Kent Tamara Rezk, Inria Joshua Sunshine, Carnegie Mellon University Steffen Zschaler, King's College London ******************************************************** 2019 is kindly supported by AOSA ******************************************************** For more information, visit http://2019.programming-conference.org From cristian.esquivias at gmail.com Sat May 5 06:50:36 2018 From: cristian.esquivias at gmail.com (Cristian Esquivias) Date: Fri, 4 May 2018 23:50:36 -0700 Subject: OutOfMemoryError on my BF interpreter In-Reply-To: References: <6435958d-15ab-aef0-f499-96ba655e5fa0@jku.at> Message-ID: Hey Christian (it's so weird typing that H ;), That command worked like a charm. Your modifications were really helpful in making my interpreter a lot faster. I can confirm that I don't get the OOME error when I build against HEAD. Yeah, I think I'm done with my BF interpreter. My original intent was to go back and work on my lisp interpreter and make it less dumb, but the API between the version of Truffle I used and 1.0.0-RC1 had changed so drastically I didn't know how to migrate. I figured writing another--even simpler--interpreter would help me understand the changes. I think I've found my bearings so now I can go update and improve my lisp interpreter. I actually read through your BF interpreter, but yours also used an old version of truffle. Since my main goal was to understand the latest APIs, I had to roll up my sleeves and implement my own while (poorly) mimicking SimpleLanguage. I'll take it as a positive sign that your interpreter and mine were similar. Thanks, everyone, for the help, Cristian On Fri, May 4, 2018 at 3:50 AM, Christian Humer wrote: > Hi Cristian, > > This command line should pick up the latest compiler: > mx -v --jdk=jvmci vm -cp > ../../bf-graal/main/target/main-1.0-SNAPSHOT.jar:../../bf-graal/lang/target/bf-1.0-SNAPSHOT.jar > cesquivias.bf.Main ../../bf-graal/examples/mandelbrot.bf > > I ran this command out of the graal compiler suite in graal/compiler folder > of the Graal repository. I checked out the bf repo at the root, hence the > ../../. > Note that the -Dtruffle.class.path.append is only a thing in GraalVM so you > need to use -cp when running with the Graal repository. > The -v command gives you some info what command it produces. Feel free to > remove it. > We are working on deploying nightly snapshots of GraalVM. That should help > you picking up the latest version in the future. > > I took the liberty to have a look at a more reasonable BF application here: > https://github.com/chumer/bf/blob/master/src/main/resources/test/primes.bf > > I was dumping this to IGV (mx igv in compiler project) and then running > using -Dgraal.Dump=. This showed me a few problems with the graph. You > should have a look at the "nodeSourcePosition" property that shows you the > origin of Graal nodes. > > I created a PR with the changes that should fix those problems: > https://github.com/cesquivias/bf-graal/pull/1 > > When I look at the IGV graph now I can see only loads and stores on your > memory int[]. Unfortunately Graal needs to materialize all the changes in > the memory as it cannot see its allocation in the OSR compilations. You > could try to materialize every memory slot into a frame slot. But I'd rather > recommend spending your time on a more interesting language like a simple > Lisp. > > BTW.: I did my own BF interpreter two years ago: > https://github.com/chumer/bf/. Looks very similar to yours :-). > > Hope this helps, > > - Christian Humer > > > > On Thu, May 3, 2018 at 8:58 AM Cristian Esquivias > wrote: >> >> @Boris >> Thanks for the tip. I saw that in the code but wasn't sure why it was >> in there. Adding a comment to explain why @TruffleBoundary is used >> could be helpful for the next person. >> >> Sadly, the change didn't make a difference. I made the changes to the >> print [1] and read [2] nodes, but I still get the same OOME at the >> same spot. >> >> [1] >> https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/PrintDataNode.java >> [2] >> https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/ReadDataNode.java >> >> >> @Thomas >> I only have two variables stored in my function's VirtualFrame: an int >> and a large (1,000-10,000 length) array of bytes >> >> The README [3] explains how to build and run the interpreter. I copied >> simplelanguage's build process so it's basically `mvn package && >> JAVA_HOME=/path/to/graalvm ./bf ./examples/mandelbrot.bf` in the root >> directory >> >> [3] https://github.com/cesquivias/bf-graal/blob/master/README.md >> >> >> @Josef >> I'm still trying to run my interpreter off of the latest source code. >> I think it built fine, but I'm not sure how to connect the compiler to >> my Java 9 JDK. I'm using mx vm but I get an error: >> >> > mx vm -XX:+UseJVMCICompiler >> > -Dtruffle.class.path.append=./bf-graal/lang/target/bf-1.0-SNAPSHOT.jar -cp >> > ./bf-graal/main/target/main-1.0-SNAPSHOT.jar cesquivias.bf.Main >> > ./bf-graal/examples/mandelbrot.bf >> Exception in thread "main" java.lang.IllegalStateException: A language >> with id 'bf' is not installed. Installed languages are: []. >> >> The compiler README [4] is a little out of date. It references the old >> graalvm repo though that repo says it's deprecated and to use the one >> under oracle org. I'm not sure if I'm calling graal right or >> referencing truffle properly. There wouldn't be maven repo that builds >> a snapshot of the graalvm that I could use instead? ;-) >> >> [4] https://github.com/oracle/graal/blob/master/compiler/README.md >> >> >> Thanks, >> Cristian >> >> On Wed, May 2, 2018 at 2:18 AM, Boris Spasojevic >> wrote: >> > Hey Christian, >> > >> > This could very well be caused by the calls to System.out.print and >> > System.in.read. >> > >> > Could you please try wrapping calls to such external methods and adding >> > a >> > @TruffleBoundary annotation? The end goal would be something like: >> > >> > public class PrintDataNode extends BFNode { >> > private final FrameSlot ptr; >> > private final FrameSlot data; >> > >> > public PrintDataNode(FrameSlot ptr, FrameSlot data) { >> > this.ptr = ptr; >> > this.data = data; >> > } >> > >> > @TruffleBoundary >> > void print(char c) { >> > System.out.print(c); >> > } >> > >> > @Override >> > public void execute(VirtualFrame frame) { >> > try { >> > print((char) ((byte[]) >> > frame.getObject(data))[frame.getInt(ptr)]); >> > } catch (FrameSlotTypeException e) { >> > e.printStackTrace(); >> > } >> > } >> > } >> > >> > You can see a similar pattern being used in Simple language >> > >> > (https://github.com/graalvm/simplelanguage/blob/master/language/src/main/java/com/oracle/truffle/sl/builtins/SLPrintlnBuiltin.java) >> > >> > Hope that helps! >> > BoriS >> > >> > >> > On 05/02/2018 10:46 AM, Cristian Esquivias wrote: >> >> >> >> Hi Josef, >> >> >> >> Thanks for the context. I'll try to build the GraalVM and see if it >> >> solves the issue. >> >> >> >> After reading the bug report, here's a little bit more info that I hope >> >> helps: >> >> >> >> * The report says the SSA register allocator could be the problem on >> >> "large inputs". I'm not sure what constitutes a large input, but my >> >> mandelbrot script creates around 11,000 nodes in total. That's >> >> probably not a huge amount, but they're all within one function since >> >> BF doesn't have the concept of functions. That's probably a little >> >> more unusual. >> >> * My interpreter doesn't put pressure on the heap. After an initial >> >> allocation of a 1,000-10,000 size int array no more allocations are >> >> done for the rest of a script's life. This is proven out by jvisualvm. >> >> The heap stays flat throughout most of the run and only spikes right >> >> before the OOME occurs. >> >> * There are very few (essentially no) opportunities for adding >> >> TruffleBoundary annotations since nearly all my nodes' execute methods >> >> are one line that do little more than fetch an item out of my array >> >> and inc/dec the value. The only calls out to external methods are >> >> calls to VirtualFrame.get [1]/set [2] some variable values, calls to >> >> stdout's print method [3] and stdin's read method [4]. A truffle >> >> expert's help would definitely be appreciated here since I'm just >> >> making an educated guess. >> >> * The OOME occurs inside a loop node. Could this OOME be caused by >> >> loop unrolling [5]? >> >> >> >> Thanks, >> >> Cristian >> >> >> >> [1]: >> >> >> >> https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/DecDataNode.java >> >> [2]: >> >> >> >> https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/DecPointerNode.java >> >> [3]: >> >> >> >> https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/PrintDataNode.java >> >> [4]: >> >> >> >> https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/ReadDataNode.java >> >> [5]: >> >> >> >> https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/BFRepeatingNode.java >> >> >> >> On Wed, May 2, 2018 at 12:37 AM, Josef Eisl wrote: >> >>> >> >>> Hi Cristian! >> >>> >> >>> Thanks for your report. We have very recently merged a change to >> >>> mitigate the OOME in this part of linear scan [1]. Unfortunately this >> >>> was after the release so it is not included in 1.0.0 RC1. You would >> >>> need >> >>> to build your own GraalVM from source to see if the change helps. >> >>> >> >>> That said, the root cause of the OOME is usually a huge compilation >> >>> unit >> >>> thrown into the compiler. For more details, see also the discussion on >> >>> the OpenJDK bug tracker [2]. >> >>> The reasons for big compilation units can me manifold. In case of >> >>> Truffle, the issues can often be solved by adding TruffleBoundaries >> >>> [3], >> >>> i.e., instructing the compiler not to inline specified methods into >> >>> the >> >>> truffle graph. Maybe a Truffle expert can give you more hints on that. >> >>> >> >>> Hope that helps, >> >>> >> >>> Josef >> >>> >> >>> [1]: >> >>> >> >>> >> >>> https://github.com/oracle/graal/commit/c0a15c562daa0338f61a6e7c22476cf145af5a66 >> >>> [2]: https://bugs.openjdk.java.net/browse/JDK-8199890 >> >>> [3]: >> >>> >> >>> >> >>> http://www.graalvm.org/truffle/javadoc/com/oracle/truffle/api/CompilerDirectives.TruffleBoundary.html >> >>> >> >>> On 01/05/18 08:24, Cristian Esquivias wrote: >> >>>> >> >>>> With the release of 1.0.0-RC1 I thought I'd come back and see how >> >>>> Graal is doing. Great job; Graal is really starting to look like the >> >>>> platform everyone should pay attention to. >> >>>> >> >>>> To get back into it, I built a Brainf*ck interpreter with Truffle >> >>>> (https://github.com/cesquivias/bf-graal). I modeled it after >> >>>> simplelanguage so it should be easy to build and run. >> >>>> >> >>>> While running the mandelbrot program, about halfway through the >> >>>> program, the VM starts throwing OutOfMemoryError exceptions (one >> >>>> pasted below). It created a zip file under dumps/ but it contains a >> >>>> .bgv file I don't what to do with and a log file that just contains >> >>>> the stacktrace. >> >>>> >> >>>> I ran jvisualvm and took a heap dump. There are ~200k long[] objects >> >>>> alone created by graal taking up ~50% of the memory. Digging through >> >>>> the references it seems to be created/retained in >> >>>> >> >>>> >> >>>> >> >>>> org.graalvm.compiler.lir.alloc.lsra.LinearScanLifetimeAnalysisPhase$$Lambda$48#1 >> >>>> [GC root - Java frame] >> >>>> >> >>>> I'd provide some file from jvisualvm but the save button is grayed >> >>>> out. I hope this is enough info. My interpreter is up on GitHub so >> >>>> feel free to test it out. >> >>>> >> >>>> I tested this both on the community & enterprise edition on Ubuntu >> >>>> 18.04 in VirtualBox. >> >>>> >> >>>> - Cristian >> >>>> >> >>>> >> >>>> Thread[TruffleCompilerThread-12,10,main]: Compilation of >> >>>> BFRepeatingNode at 222b298d failed: java.lang.OutOfMemoryError: >> >>>> Java >> >>>> heap space >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.lir.util.IndexedValueMap.(IndexedValueMap.java:55) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.lir.dfa.RegStackValueSet.(RegStackValueSet.java:62) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:70) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:46) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.lir.dfa.LocationMarker.processBlock(LocationMarker.java:107) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.lir.dfa.LocationMarker.build(LocationMarker.java:81) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:51) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:47) >> >>>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) >> >>>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.lir.phases.LIRPhaseSuite.run(LIRPhaseSuite.java:96) >> >>>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) >> >>>> at org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.core.GraalCompiler.emitLowLevel(GraalCompiler.java:367) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.core.GraalCompiler.emitLIR0(GraalCompiler.java:336) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.core.GraalCompiler.emitLIR(GraalCompiler.java:295) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.core.GraalCompiler.emitBackEnd(GraalCompiler.java:275) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.core.GraalCompiler.compile(GraalCompiler.java:175) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.core.GraalCompiler.compileGraph(GraalCompiler.java:160) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compilePEGraph(TruffleCompilerImpl.java:445) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compileAST(TruffleCompilerImpl.java:391) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:544) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:493) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.core.CompilationWrapper.run(CompilationWrapper.java:167) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.doCompile(TruffleCompilerImpl.java:222) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime.doCompile(GraalTruffleRuntime.java:679) >> >>>> at >> >>>> >> >>>> org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime$1.run(GraalTruffleRuntime.java:745) >> >>>> at >> >>>> >> >>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> >>>> at >> >>>> >> >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >> >>>> at >> >>>> >> >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >> >>>> at java.lang.Thread.run(Thread.java:748) >> >>>> To disable compilation failure notifications, set >> >>>> CompilationFailureAction to Silent (e.g., >> >>>> -Dgraal.CompilationFailureAction=Silent). >> >>>> To print a message for a compilation failure without retrying the >> >>>> compilation, set CompilationFailureAction to Print (e.g., >> >>>> -Dgraal.CompilationFailureAction=Print). >> >>>> Retrying compilation of BFRepeatingNode at 222b298d >> >>>> >> > From christian.humer at gmail.com Sat May 5 10:53:25 2018 From: christian.humer at gmail.com (Christian Humer) Date: Sat, 05 May 2018 10:53:25 +0000 Subject: OutOfMemoryError on my BF interpreter In-Reply-To: References: <6435958d-15ab-aef0-f499-96ba655e5fa0@jku.at> Message-ID: Hey Cristian (it's so weird without that H ;), Glad it helped. We recommed migrating one Truffle version at a time. That makes it easier to pick up the changes. I wrote a paragraph about that in the readme that is easy to miss: https://github.com/oracle/graal/tree/master/truffle#compatibility Thanks for your feedback. - Christian Humer On Sat, May 5, 2018 at 8:50 AM Cristian Esquivias < cristian.esquivias at gmail.com> wrote: > Hey Christian (it's so weird typing that H ;), > > That command worked like a charm. Your modifications were really > helpful in making my interpreter a lot faster. I can confirm that I > don't get the OOME error when I build against HEAD. > > Yeah, I think I'm done with my BF interpreter. My original intent was > to go back and work on my lisp interpreter and make it less dumb, but > the API between the version of Truffle I used and 1.0.0-RC1 had > changed so drastically I didn't know how to migrate. I figured writing > another--even simpler--interpreter would help me understand the > changes. I think I've found my bearings so now I can go update and > improve my lisp interpreter. > > I actually read through your BF interpreter, but yours also used an > old version of truffle. Since my main goal was to understand the > latest APIs, I had to roll up my sleeves and implement my own while > (poorly) mimicking SimpleLanguage. I'll take it as a positive sign > that your interpreter and mine were similar. > > Thanks, everyone, for the help, > Cristian > > On Fri, May 4, 2018 at 3:50 AM, Christian Humer > wrote: > > Hi Cristian, > > > > This command line should pick up the latest compiler: > > mx -v --jdk=jvmci vm -cp > > > ../../bf-graal/main/target/main-1.0-SNAPSHOT.jar:../../bf-graal/lang/target/bf-1.0-SNAPSHOT.jar > > cesquivias.bf.Main ../../bf-graal/examples/mandelbrot.bf > > > > I ran this command out of the graal compiler suite in graal/compiler > folder > > of the Graal repository. I checked out the bf repo at the root, hence the > > ../../. > > Note that the -Dtruffle.class.path.append is only a thing in GraalVM so > you > > need to use -cp when running with the Graal repository. > > The -v command gives you some info what command it produces. Feel free to > > remove it. > > We are working on deploying nightly snapshots of GraalVM. That should > help > > you picking up the latest version in the future. > > > > I took the liberty to have a look at a more reasonable BF application > here: > > > https://github.com/chumer/bf/blob/master/src/main/resources/test/primes.bf > > > > I was dumping this to IGV (mx igv in compiler project) and then running > > using -Dgraal.Dump=. This showed me a few problems with the graph. You > > should have a look at the "nodeSourcePosition" property that shows you > the > > origin of Graal nodes. > > > > I created a PR with the changes that should fix those problems: > > https://github.com/cesquivias/bf-graal/pull/1 > > > > When I look at the IGV graph now I can see only loads and stores on your > > memory int[]. Unfortunately Graal needs to materialize all the changes in > > the memory as it cannot see its allocation in the OSR compilations. You > > could try to materialize every memory slot into a frame slot. But I'd > rather > > recommend spending your time on a more interesting language like a simple > > Lisp. > > > > BTW.: I did my own BF interpreter two years ago: > > https://github.com/chumer/bf/. Looks very similar to yours :-). > > > > Hope this helps, > > > > - Christian Humer > > > > > > > > On Thu, May 3, 2018 at 8:58 AM Cristian Esquivias > > wrote: > >> > >> @Boris > >> Thanks for the tip. I saw that in the code but wasn't sure why it was > >> in there. Adding a comment to explain why @TruffleBoundary is used > >> could be helpful for the next person. > >> > >> Sadly, the change didn't make a difference. I made the changes to the > >> print [1] and read [2] nodes, but I still get the same OOME at the > >> same spot. > >> > >> [1] > >> > https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/PrintDataNode.java > >> [2] > >> > https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/ReadDataNode.java > >> > >> > >> @Thomas > >> I only have two variables stored in my function's VirtualFrame: an int > >> and a large (1,000-10,000 length) array of bytes > >> > >> The README [3] explains how to build and run the interpreter. I copied > >> simplelanguage's build process so it's basically `mvn package && > >> JAVA_HOME=/path/to/graalvm ./bf ./examples/mandelbrot.bf` in the root > >> directory > >> > >> [3] https://github.com/cesquivias/bf-graal/blob/master/README.md > >> > >> > >> @Josef > >> I'm still trying to run my interpreter off of the latest source code. > >> I think it built fine, but I'm not sure how to connect the compiler to > >> my Java 9 JDK. I'm using mx vm but I get an error: > >> > >> > mx vm -XX:+UseJVMCICompiler > >> > > -Dtruffle.class.path.append=./bf-graal/lang/target/bf-1.0-SNAPSHOT.jar -cp > >> > ./bf-graal/main/target/main-1.0-SNAPSHOT.jar cesquivias.bf.Main > >> > ./bf-graal/examples/mandelbrot.bf > >> Exception in thread "main" java.lang.IllegalStateException: A language > >> with id 'bf' is not installed. Installed languages are: []. > >> > >> The compiler README [4] is a little out of date. It references the old > >> graalvm repo though that repo says it's deprecated and to use the one > >> under oracle org. I'm not sure if I'm calling graal right or > >> referencing truffle properly. There wouldn't be maven repo that builds > >> a snapshot of the graalvm that I could use instead? ;-) > >> > >> [4] https://github.com/oracle/graal/blob/master/compiler/README.md > >> > >> > >> Thanks, > >> Cristian > >> > >> On Wed, May 2, 2018 at 2:18 AM, Boris Spasojevic > >> wrote: > >> > Hey Christian, > >> > > >> > This could very well be caused by the calls to System.out.print and > >> > System.in.read. > >> > > >> > Could you please try wrapping calls to such external methods and > adding > >> > a > >> > @TruffleBoundary annotation? The end goal would be something like: > >> > > >> > public class PrintDataNode extends BFNode { > >> > private final FrameSlot ptr; > >> > private final FrameSlot data; > >> > > >> > public PrintDataNode(FrameSlot ptr, FrameSlot data) { > >> > this.ptr = ptr; > >> > this.data = data; > >> > } > >> > > >> > @TruffleBoundary > >> > void print(char c) { > >> > System.out.print(c); > >> > } > >> > > >> > @Override > >> > public void execute(VirtualFrame frame) { > >> > try { > >> > print((char) ((byte[]) > >> > frame.getObject(data))[frame.getInt(ptr)]); > >> > } catch (FrameSlotTypeException e) { > >> > e.printStackTrace(); > >> > } > >> > } > >> > } > >> > > >> > You can see a similar pattern being used in Simple language > >> > > >> > ( > https://github.com/graalvm/simplelanguage/blob/master/language/src/main/java/com/oracle/truffle/sl/builtins/SLPrintlnBuiltin.java > ) > >> > > >> > Hope that helps! > >> > BoriS > >> > > >> > > >> > On 05/02/2018 10:46 AM, Cristian Esquivias wrote: > >> >> > >> >> Hi Josef, > >> >> > >> >> Thanks for the context. I'll try to build the GraalVM and see if it > >> >> solves the issue. > >> >> > >> >> After reading the bug report, here's a little bit more info that I > hope > >> >> helps: > >> >> > >> >> * The report says the SSA register allocator could be the problem on > >> >> "large inputs". I'm not sure what constitutes a large input, but my > >> >> mandelbrot script creates around 11,000 nodes in total. That's > >> >> probably not a huge amount, but they're all within one function since > >> >> BF doesn't have the concept of functions. That's probably a little > >> >> more unusual. > >> >> * My interpreter doesn't put pressure on the heap. After an initial > >> >> allocation of a 1,000-10,000 size int array no more allocations are > >> >> done for the rest of a script's life. This is proven out by > jvisualvm. > >> >> The heap stays flat throughout most of the run and only spikes right > >> >> before the OOME occurs. > >> >> * There are very few (essentially no) opportunities for adding > >> >> TruffleBoundary annotations since nearly all my nodes' execute > methods > >> >> are one line that do little more than fetch an item out of my array > >> >> and inc/dec the value. The only calls out to external methods are > >> >> calls to VirtualFrame.get [1]/set [2] some variable values, calls to > >> >> stdout's print method [3] and stdin's read method [4]. A truffle > >> >> expert's help would definitely be appreciated here since I'm just > >> >> making an educated guess. > >> >> * The OOME occurs inside a loop node. Could this OOME be caused by > >> >> loop unrolling [5]? > >> >> > >> >> Thanks, > >> >> Cristian > >> >> > >> >> [1]: > >> >> > >> >> > https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/DecDataNode.java > >> >> [2]: > >> >> > >> >> > https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/DecPointerNode.java > >> >> [3]: > >> >> > >> >> > https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/PrintDataNode.java > >> >> [4]: > >> >> > >> >> > https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/ReadDataNode.java > >> >> [5]: > >> >> > >> >> > https://github.com/cesquivias/bf-graal/blob/master/lang/src/main/java/cesquivias/bf/BFRepeatingNode.java > >> >> > >> >> On Wed, May 2, 2018 at 12:37 AM, Josef Eisl > wrote: > >> >>> > >> >>> Hi Cristian! > >> >>> > >> >>> Thanks for your report. We have very recently merged a change to > >> >>> mitigate the OOME in this part of linear scan [1]. Unfortunately > this > >> >>> was after the release so it is not included in 1.0.0 RC1. You would > >> >>> need > >> >>> to build your own GraalVM from source to see if the change helps. > >> >>> > >> >>> That said, the root cause of the OOME is usually a huge compilation > >> >>> unit > >> >>> thrown into the compiler. For more details, see also the discussion > on > >> >>> the OpenJDK bug tracker [2]. > >> >>> The reasons for big compilation units can me manifold. In case of > >> >>> Truffle, the issues can often be solved by adding TruffleBoundaries > >> >>> [3], > >> >>> i.e., instructing the compiler not to inline specified methods into > >> >>> the > >> >>> truffle graph. Maybe a Truffle expert can give you more hints on > that. > >> >>> > >> >>> Hope that helps, > >> >>> > >> >>> Josef > >> >>> > >> >>> [1]: > >> >>> > >> >>> > >> >>> > https://github.com/oracle/graal/commit/c0a15c562daa0338f61a6e7c22476cf145af5a66 > >> >>> [2]: https://bugs.openjdk.java.net/browse/JDK-8199890 > >> >>> [3]: > >> >>> > >> >>> > >> >>> > http://www.graalvm.org/truffle/javadoc/com/oracle/truffle/api/CompilerDirectives.TruffleBoundary.html > >> >>> > >> >>> On 01/05/18 08:24, Cristian Esquivias wrote: > >> >>>> > >> >>>> With the release of 1.0.0-RC1 I thought I'd come back and see how > >> >>>> Graal is doing. Great job; Graal is really starting to look like > the > >> >>>> platform everyone should pay attention to. > >> >>>> > >> >>>> To get back into it, I built a Brainf*ck interpreter with Truffle > >> >>>> (https://github.com/cesquivias/bf-graal). I modeled it after > >> >>>> simplelanguage so it should be easy to build and run. > >> >>>> > >> >>>> While running the mandelbrot program, about halfway through the > >> >>>> program, the VM starts throwing OutOfMemoryError exceptions (one > >> >>>> pasted below). It created a zip file under dumps/ but it contains a > >> >>>> .bgv file I don't what to do with and a log file that just contains > >> >>>> the stacktrace. > >> >>>> > >> >>>> I ran jvisualvm and took a heap dump. There are ~200k long[] > objects > >> >>>> alone created by graal taking up ~50% of the memory. Digging > through > >> >>>> the references it seems to be created/retained in > >> >>>> > >> >>>> > >> >>>> > >> >>>> > org.graalvm.compiler.lir.alloc.lsra.LinearScanLifetimeAnalysisPhase$$Lambda$48#1 > >> >>>> [GC root - Java frame] > >> >>>> > >> >>>> I'd provide some file from jvisualvm but the save button is grayed > >> >>>> out. I hope this is enough info. My interpreter is up on GitHub so > >> >>>> feel free to test it out. > >> >>>> > >> >>>> I tested this both on the community & enterprise edition on Ubuntu > >> >>>> 18.04 in VirtualBox. > >> >>>> > >> >>>> - Cristian > >> >>>> > >> >>>> > >> >>>> Thread[TruffleCompilerThread-12,10,main]: Compilation of > >> >>>> BFRepeatingNode at 222b298d failed: java.lang.OutOfMemoryError: > >> >>>> Java > >> >>>> heap space > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.lir.util.IndexedValueMap.(IndexedValueMap.java:55) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.lir.dfa.RegStackValueSet.(RegStackValueSet.java:62) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:70) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.lir.dfa.RegStackValueSet.copy(RegStackValueSet.java:46) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.lir.dfa.LocationMarker.processBlock(LocationMarker.java:107) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.lir.dfa.LocationMarker.build(LocationMarker.java:81) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:51) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.lir.dfa.LocationMarkerPhase.run(LocationMarkerPhase.java:47) > >> >>>> at > org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) > >> >>>> at > org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.lir.phases.LIRPhaseSuite.run(LIRPhaseSuite.java:96) > >> >>>> at > org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:115) > >> >>>> at > org.graalvm.compiler.lir.phases.LIRPhase.apply(LIRPhase.java:107) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.core.GraalCompiler.emitLowLevel(GraalCompiler.java:367) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.core.GraalCompiler.emitLIR0(GraalCompiler.java:336) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.core.GraalCompiler.emitLIR(GraalCompiler.java:295) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.core.GraalCompiler.emitBackEnd(GraalCompiler.java:275) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.core.GraalCompiler.compile(GraalCompiler.java:175) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.core.GraalCompiler.compileGraph(GraalCompiler.java:160) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compilePEGraph(TruffleCompilerImpl.java:445) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compileAST(TruffleCompilerImpl.java:391) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:544) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:493) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.core.CompilationWrapper.run(CompilationWrapper.java:167) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.doCompile(TruffleCompilerImpl.java:222) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime.doCompile(GraalTruffleRuntime.java:679) > >> >>>> at > >> >>>> > >> >>>> > org.graalvm.compiler.truffle.runtime.GraalTruffleRuntime$1.run(GraalTruffleRuntime.java:745) > >> >>>> at > >> >>>> > >> >>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > >> >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) > >> >>>> at > >> >>>> > >> >>>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > >> >>>> at > >> >>>> > >> >>>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > >> >>>> at java.lang.Thread.run(Thread.java:748) > >> >>>> To disable compilation failure notifications, set > >> >>>> CompilationFailureAction to Silent (e.g., > >> >>>> -Dgraal.CompilationFailureAction=Silent). > >> >>>> To print a message for a compilation failure without retrying the > >> >>>> compilation, set CompilationFailureAction to Print (e.g., > >> >>>> -Dgraal.CompilationFailureAction=Print). > >> >>>> Retrying compilation of BFRepeatingNode at 222b298d > >> >>>> > >> > > From tbaldridge at gmail.com Sun May 6 23:21:20 2018 From: tbaldridge at gmail.com (Timothy Baldridge) Date: Sun, 6 May 2018 17:21:20 -0600 Subject: Developing with native-image/Truffle on OSX Message-ID: Been planning to do a dive into SubstrateVM and Truffle, having written several interpreters in PyPy's toolchain. However, I can't seem to find any tutorials on how to get GrallVM, native-image, and Truffle all working on OSX. Is it possible today or do I need to dig up a Linux box? I don't mind building stuff from source, but the "Getting started" docs on Graal were a bit sparse, saying run `mx vm ` to start Graal, doesn't help me much. From chris.seaton at oracle.com Sun May 6 23:27:07 2018 From: chris.seaton at oracle.com (Chris Seaton) Date: Mon, 7 May 2018 00:27:07 +0100 Subject: Developing with native-image/Truffle on OSX In-Reply-To: References: Message-ID: <83003C53-C45A-4949-AFE5-45803BF5845E@oracle.com> Most of us develop on macOS, so it?s definitely a fully supported platform for development. If you want to get these components working as easily as possible, just get GraalVM EE macOS build from OTN and you will have everything you need. http://www.graalvm.org/downloads/ If you want to build from source, some of them are easier than others. I recommend you try the standalone SimpleLanguage first. This shows you how you might work if you were writing your own language. https://github.com/graalvm/simplelanguage To build the rest of Graal from source is fairly straight forward. Building a GraalVM tarball as per the release might take a bit of work but is definitely doable on macOS. Are those pointers helpful? Chris > On 7 May 2018, at 00:21, Timothy Baldridge wrote: > > Been planning to do a dive into SubstrateVM and Truffle, having written > several interpreters in PyPy's toolchain. However, I can't seem to find any > tutorials on how to get GrallVM, native-image, and Truffle all working on > OSX. Is it possible today or do I need to dig up a Linux box? > > I don't mind building stuff from source, but the "Getting started" docs on > Graal were a bit sparse, saying run `mx vm ` to start Graal, doesn't help > me much. From tbaldridge at gmail.com Tue May 8 23:08:55 2018 From: tbaldridge at gmail.com (Timothy Baldridge) Date: Tue, 8 May 2018 17:08:55 -0600 Subject: What's the entry point for Shape/DynamicObject? Message-ID: I've been looking into the code behind Shape, DynamicObject, Layouts, etc. And I can't seem to find what the top level object is. What should I be calling to build a shape implementation? Should I be using object.basic.ShapeBasic or something else? Also is there an example language I could look at that uses these classes? Thanks, Timothy From chris.seaton at oracle.com Tue May 8 23:17:24 2018 From: chris.seaton at oracle.com (Chris Seaton) Date: Wed, 9 May 2018 00:17:24 +0100 Subject: What's the entry point for Shape/DynamicObject? In-Reply-To: References: Message-ID: <4ED326DC-147A-4A97-A840-E05BFE812BE6@oracle.com> The entry point looks something like this. Layout layout = Layout.newLayout()?...build() Shape shape = layout.createShape().addProperty()?. // each .addProperty returns a new shape also containing that property DynamicObject object = shape.newInstance() Chris > On 9 May 2018, at 00:08, Timothy Baldridge wrote: > > I've been looking into the code behind Shape, DynamicObject, Layouts, etc. > And I can't seem to find what the top level object is. What should I be > calling to build a shape implementation? Should I be using > object.basic.ShapeBasic or something else? Also is there an example > language I could look at that uses these classes? > > Thanks, > > Timothy From doug.simon at oracle.com Thu May 17 15:32:33 2018 From: doug.simon at oracle.com (Doug Simon) Date: Thu, 17 May 2018 17:32:33 +0200 Subject: JVMCI 0.43 released Message-ID: <81C634E3-4F2F-4387-AD73-F4C0CD733DE3@oracle.com> JVMCI 0.43 contains the following changes: [GR-9605] Merge with 8u171. [GR-8129] Add simple access to the CompileLog. [GR-9490] Make jvmci build work on Windows. [GR-9523] Add null checks during construction of HotSpotVMConfigStore. [GR-9336] Require Override annotations. [GR-9209] Fix problem with virtual constructors and new warning. The OracleJDK based binaries will soon be available at: http://www.oracle.com/technetwork/oracle-labs/program-languages/downloads/index.html The OpenJDK based binary is at: https://github.com/graalvm/openjdk8-jvmci-builder/releases/tag/jvmci-0.43 -Doug From aph at redhat.com Fri May 25 15:38:31 2018 From: aph at redhat.com (Andrew Haley) Date: Fri, 25 May 2018 16:38:31 +0100 Subject: Graal build failed Message-ID: <15126a1b-9742-2031-0d84-328d2c9d1127@redhat.com> This is today's JDK, today's Graal: mustang-b0:compiler $ mx build Compiling com.oracle.truffle.api.interop with javac-daemon... [/local/graal/graal/truffle/mxbuild/src/com.oracle.truffle.api.interop/bin/com/oracle/truffle/api/interop/IsInstantiable.class does not exist] Compiling org.graalvm.compiler.lir with javac-daemon... [/local/graal/graal/compiler/mxbuild/src/org.graalvm.compiler.lir/bin/org/graalvm/compiler/lir/CompositeValueClass.class does not exist] Compiling com.oracle.truffle.api.interop with javac-daemon failed 1 build tasks failed http://hg.openjdk.java.net/jdk/jdk https://github.com/graalvm/graal.git Any ideas? -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Fri May 25 15:45:16 2018 From: aph at redhat.com (Andrew Haley) Date: Fri, 25 May 2018 16:45:16 +0100 Subject: Graal build failed In-Reply-To: <15126a1b-9742-2031-0d84-328d2c9d1127@redhat.com> References: <15126a1b-9742-2031-0d84-328d2c9d1127@redhat.com> Message-ID: <80927f2a-992a-3f3d-0d20-1d5f845286f5@redhat.com> On 05/25/2018 04:38 PM, Andrew Haley wrote: > Any ideas? This may be AArch64-related: more later. Sorry for the noise. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Fri May 25 17:20:10 2018 From: aph at redhat.com (Andrew Haley) Date: Fri, 25 May 2018 18:20:10 +0100 Subject: Please update Graal in OpenJDK Message-ID: Two reasons: Firstly, AAcrh64 AOT is in there, and I'd like it in JDK now. Secondly, mx updategraalinopenjdk no longer works: I think some of the build files in OpenJDK need to be changed because Graal has been refactored. Thank you very much. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From doug.simon at oracle.com Sat May 26 08:41:42 2018 From: doug.simon at oracle.com (Doug Simon) Date: Sat, 26 May 2018 10:41:42 +0200 Subject: Please update Graal in OpenJDK In-Reply-To: References: Message-ID: <8B17EC79-C67D-43AE-951E-DA4D917CBC55@oracle.com> I've added a patch to https://bugs.openjdk.java.net/browse/JDK-8202670 that needs to be applied after running `mx updategraalinopenjdk`. -Doug > On 25 May 2018, at 19:20, Andrew Haley wrote: > > Two reasons: > > Firstly, AAcrh64 AOT is in there, and I'd like it in JDK now. > > Secondly, mx updategraalinopenjdk no longer works: I think some of the > build files in OpenJDK need to be changed because Graal has been > refactored. > > Thank you very much. > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Sat May 26 09:03:30 2018 From: aph at redhat.com (Andrew Haley) Date: Sat, 26 May 2018 10:03:30 +0100 Subject: Please update Graal in OpenJDK In-Reply-To: <8B17EC79-C67D-43AE-951E-DA4D917CBC55@oracle.com> References: <8B17EC79-C67D-43AE-951E-DA4D917CBC55@oracle.com> Message-ID: On 05/26/2018 09:41 AM, Doug Simon wrote: > I've added a patch to https://bugs.openjdk.java.net/browse/JDK-8202670 that needs to be applied after running `mx updategraalinopenjdk`. OK, I'll try that, Thanks. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From juan.fumero at manchester.ac.uk Mon May 28 08:14:57 2018 From: juan.fumero at manchester.ac.uk (Juan Fumero) Date: Mon, 28 May 2018 10:14:57 +0200 Subject: Java Local annotations and Graal Inlining Message-ID: <1527495297.5613.7.camel@manchester.ac.uk> Hi all, I have a question concerning local Java annotations and the inlining phase in Graal.? I want to get Java annotations associated with local variables (runtime annotations). The way we currently do is through the ResolvedJavaMethod and the getLocalAnnotations method (we based on Graal-0.22). I wanted to reproduce this example using the last Graal but this method does not exist anymore.? Currently, we can see local annotations when we directly compile the method that contains the annotations. However, when the method contains a methodInvoke, after inlining we do not see any local annotations any more. Is this due to an error during inlining?? To clarify, here I show you a snippet of what we are trying to do.? Our goal is to associate a Graal-IR node with a Java annotation. In this case, a PhiNode (i var). @Target({ ElementType.LOCAL_VARIABLE, ElementType.TYPE}) @Retention(RetentionPolicy.RUNTIME) public @interface Local { } public void foo(...) { ???bar(...) } public void bar(...) { ???for (@Local int i = 0, i ... ) { ???} } If I build the Graal-IR for the method `bar`, I see @Local for the PhiNode(i var). However, when I compile the method `foo`, I can the inlined loop in `foo` after inlining but I do not see @Local anymore. The real code is more complicated, `foo` is from an abstract class and bar is directly the user code. Our compiler has to start compiling from method `foo`. Any pointers, suggestions or work around will be appreciated.? Thanks, Juan From doug.simon at oracle.com Mon May 28 15:40:28 2018 From: doug.simon at oracle.com (Doug Simon) Date: Mon, 28 May 2018 17:40:28 +0200 Subject: Java Local annotations and Graal Inlining In-Reply-To: <1527495297.5613.7.camel@manchester.ac.uk> References: <1527495297.5613.7.camel@manchester.ac.uk> Message-ID: Hi Juan, I'm a little lost. I don't recall ResolvedJavaMethod ever having a getLocalAnnotations method. Standard Java reflection doesn't provide this either. I think you can only get at these by parsing the class files. -Doug > On 28 May 2018, at 10:14, Juan Fumero wrote: > > Hi all, > > I have a question concerning local Java annotations and the inlining > phase in Graal. > > I want to get Java annotations associated with local variables (runtime > annotations). The way we currently do is through the ResolvedJavaMethod > and the getLocalAnnotations method (we based on Graal-0.22). I wanted > to reproduce this example using the last Graal but this method does not > exist anymore. > > Currently, we can see local annotations when we directly compile the > method that contains the annotations. However, when the method contains > a methodInvoke, after inlining we do not see any local annotations any > more. Is this due to an error during inlining? > > To clarify, here I show you a snippet of what we are trying to do. > Our goal is to associate a Graal-IR node with a Java annotation. In > this case, a PhiNode (i var). > > > @Target({ ElementType.LOCAL_VARIABLE, ElementType.TYPE}) > @Retention(RetentionPolicy.RUNTIME) > public @interface Local { > } > > > public void foo(...) { > bar(...) > } > > public void bar(...) { > for (@Local int i = 0, i ... ) { > > } > } > > > If I build the Graal-IR for the method `bar`, I see @Local for the > PhiNode(i var). However, when I compile the method `foo`, I can the > inlined loop in `foo` after inlining but I do not see @Local anymore. > The real code is more complicated, `foo` is from an abstract class and > bar is directly the user code. Our compiler has to start compiling from > method `foo`. > > > Any pointers, suggestions or work around will be appreciated. > > > Thanks, > Juan > From aph at redhat.com Tue May 29 12:02:14 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 29 May 2018 13:02:14 +0100 Subject: Please update Graal in OpenJDK In-Reply-To: References: <8B17EC79-C67D-43AE-951E-DA4D917CBC55@oracle.com> Message-ID: <27553ef2-4587-7093-c5b7-f52813109f2d@redhat.com> On 05/26/2018 10:03 AM, Andrew Haley wrote: > On 05/26/2018 09:41 AM, Doug Simon wrote: >> I've added a patch to https://bugs.openjdk.java.net/browse/JDK-8202670 that needs to be applied after running `mx updategraalinopenjdk`. > > OK, I'll try that, Thanks. > Still a few problems: Creating symlink jdk/lib/ct.sym /local/jdk-jdk/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements.jdk9.test/src/org/graalvm/compiler/replacements/jdk9/UnsafeReplacementsTest.java:28: error: package org.graalvm.compiler.api.test does not exist import org.graalvm.compiler.api.test.Graal; ^ /local/jdk-jdk/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements.jdk9.test/src/org/graalvm/compiler/replacements/jdk9/UnsafeReplacementsTest.java:29: error: package org.graalvm.compiler.replacements.test does not exist import org.graalvm.compiler.replacements.test.MethodSubstitutionTest; ^ ... Did this work for you? -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From doug.simon at oracle.com Tue May 29 12:27:58 2018 From: doug.simon at oracle.com (Doug Simon) Date: Tue, 29 May 2018 14:27:58 +0200 Subject: Please update Graal in OpenJDK In-Reply-To: <27553ef2-4587-7093-c5b7-f52813109f2d@redhat.com> References: <8B17EC79-C67D-43AE-951E-DA4D917CBC55@oracle.com> <27553ef2-4587-7093-c5b7-f52813109f2d@redhat.com> Message-ID: > On 29 May 2018, at 14:02, Andrew Haley wrote: > > On 05/26/2018 10:03 AM, Andrew Haley wrote: >> On 05/26/2018 09:41 AM, Doug Simon wrote: >>> I've added a patch to https://bugs.openjdk.java.net/browse/JDK-8202670 that needs to be applied after running `mx updategraalinopenjdk`. >> >> OK, I'll try that, Thanks. >> > > Still a few problems: > > Creating symlink jdk/lib/ct.sym > /local/jdk-jdk/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements.jdk9.test/src/org/graalvm/compiler/replacements/jdk9/UnsafeReplacementsTest.java:28: error: package org.graalvm.compiler.api.test does not exist > import org.graalvm.compiler.api.test.Graal; > ^ > /local/jdk-jdk/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements.jdk9.test/src/org/graalvm/compiler/replacements/jdk9/UnsafeReplacementsTest.java:29: error: package org.graalvm.compiler.replacements.test does not exist > import org.graalvm.compiler.replacements.test.MethodSubstitutionTest; > ^ > ... > > Did this work for you? Looks like the patch didn't apply cleanly as it excludes org.graalvm.compiler.replacements.test.jdk9: diff -r f84ae8aa5d88 make/CompileJavaModules.gmk --- a/make/CompileJavaModules.gmk Sat May 26 03:11:50 2018 -0400 +++ b/make/CompileJavaModules.gmk Sat May 26 10:36:35 2018 +0200 @@ -440,11 +440,13 @@ jdk.internal.vm.compiler_EXCLUDES += \ jdk.internal.vm.compiler.collections.test \ + org.graalvm.compiler.processor \ org.graalvm.compiler.core.match.processor \ org.graalvm.compiler.nodeinfo.processor \ org.graalvm.compiler.options.processor \ org.graalvm.compiler.serviceprovider.processor \ - org.graalvm.compiler.replacements.verifier \ + org.graalvm.compiler.replacements.processor \ + org.graalvm.compiler.replacements.test.jdk9 \ org.graalvm.compiler.api.directives.test \ org.graalvm.compiler.api.test \ org.graalvm.compiler.asm.aarch64.test \ -Doug From aph at redhat.com Tue May 29 13:07:03 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 29 May 2018 14:07:03 +0100 Subject: Please update Graal in OpenJDK In-Reply-To: References: <8B17EC79-C67D-43AE-951E-DA4D917CBC55@oracle.com> <27553ef2-4587-7093-c5b7-f52813109f2d@redhat.com> Message-ID: <378474e3-472b-5dcd-e19d-d5ffb6825c53@redhat.com> On 05/29/2018 01:27 PM, Doug Simon wrote: > > >> On 29 May 2018, at 14:02, Andrew Haley wrote: >> >> On 05/26/2018 10:03 AM, Andrew Haley wrote: >>> On 05/26/2018 09:41 AM, Doug Simon wrote: >>>> I've added a patch to https://bugs.openjdk.java.net/browse/JDK-8202670 that needs to be applied after running `mx updategraalinopenjdk`. >>> >>> OK, I'll try that, Thanks. >>> >> >> Still a few problems: >> >> Creating symlink jdk/lib/ct.sym >> /local/jdk-jdk/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements.jdk9.test/src/org/graalvm/compiler/replacements/jdk9/UnsafeReplacementsTest.java:28: error: package org.graalvm.compiler.api.test does not exist >> import org.graalvm.compiler.api.test.Graal; >> ^ >> /local/jdk-jdk/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements.jdk9.test/src/org/graalvm/compiler/replacements/jdk9/UnsafeReplacementsTest.java:29: error: package org.graalvm.compiler.replacements.test does not exist >> import org.graalvm.compiler.replacements.test.MethodSubstitutionTest; >> ^ >> ... >> >> Did this work for you? > > Looks like the patch didn't apply cleanly as it excludes org.graalvm.compiler.replacements.test.jdk9: > > diff -r f84ae8aa5d88 make/CompileJavaModules.gmk > --- a/make/CompileJavaModules.gmk Sat May 26 03:11:50 2018 -0400 > +++ b/make/CompileJavaModules.gmk Sat May 26 10:36:35 2018 +0200 > @@ -440,11 +440,13 @@ > > jdk.internal.vm.compiler_EXCLUDES += \ > jdk.internal.vm.compiler.collections.test \ > + org.graalvm.compiler.processor \ > org.graalvm.compiler.core.match.processor \ > org.graalvm.compiler.nodeinfo.processor \ > org.graalvm.compiler.options.processor \ > org.graalvm.compiler.serviceprovider.processor \ > - org.graalvm.compiler.replacements.verifier \ > + org.graalvm.compiler.replacements.processor \ > + org.graalvm.compiler.replacements.test.jdk9 \ > org.graalvm.compiler.api.directives.test \ > org.graalvm.compiler.api.test \ > org.graalvm.compiler.asm.aarch64.test \ This is what the file looks like after I patched it: jdk.internal.vm.compiler_EXCLUDES += \ jdk.internal.vm.compiler.collections.test \ org.graalvm.compiler.processor \ org.graalvm.compiler.core.match.processor \ org.graalvm.compiler.nodeinfo.processor \ org.graalvm.compiler.options.processor \ org.graalvm.compiler.serviceprovider.processor \ org.graalvm.compiler.replacements.processor \ org.graalvm.compiler.replacements.test.jdk9 \ org.graalvm.compiler.api.directives.test \ ... so that patch definitely worked. But the package is called org.graalvm.compiler.replacements.jdk9.test, not org.graalvm.compiler.replacements.test.jdk9 -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From doug.simon at oracle.com Tue May 29 13:40:34 2018 From: doug.simon at oracle.com (Doug Simon) Date: Tue, 29 May 2018 15:40:34 +0200 Subject: Please update Graal in OpenJDK In-Reply-To: <378474e3-472b-5dcd-e19d-d5ffb6825c53@redhat.com> References: <8B17EC79-C67D-43AE-951E-DA4D917CBC55@oracle.com> <27553ef2-4587-7093-c5b7-f52813109f2d@redhat.com> <378474e3-472b-5dcd-e19d-d5ffb6825c53@redhat.com> Message-ID: <247D5C6D-BCDE-4478-AAAA-2C57A1C73B0E@oracle.com> > On 29 May 2018, at 15:07, Andrew Haley wrote: > > On 05/29/2018 01:27 PM, Doug Simon wrote: >> >> >>> On 29 May 2018, at 14:02, Andrew Haley wrote: >>> >>> On 05/26/2018 10:03 AM, Andrew Haley wrote: >>>> On 05/26/2018 09:41 AM, Doug Simon wrote: >>>>> I've added a patch to https://bugs.openjdk.java.net/browse/JDK-8202670 that needs to be applied after running `mx updategraalinopenjdk`. >>>> >>>> OK, I'll try that, Thanks. >>>> >>> >>> Still a few problems: >>> >>> Creating symlink jdk/lib/ct.sym >>> /local/jdk-jdk/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements.jdk9.test/src/org/graalvm/compiler/replacements/jdk9/UnsafeReplacementsTest.java:28: error: package org.graalvm.compiler.api.test does not exist >>> import org.graalvm.compiler.api.test.Graal; >>> ^ >>> /local/jdk-jdk/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements.jdk9.test/src/org/graalvm/compiler/replacements/jdk9/UnsafeReplacementsTest.java:29: error: package org.graalvm.compiler.replacements.test does not exist >>> import org.graalvm.compiler.replacements.test.MethodSubstitutionTest; >>> ^ >>> ... >>> >>> Did this work for you? >> >> Looks like the patch didn't apply cleanly as it excludes org.graalvm.compiler.replacements.test.jdk9: >> >> diff -r f84ae8aa5d88 make/CompileJavaModules.gmk >> --- a/make/CompileJavaModules.gmk Sat May 26 03:11:50 2018 -0400 >> +++ b/make/CompileJavaModules.gmk Sat May 26 10:36:35 2018 +0200 >> @@ -440,11 +440,13 @@ >> >> jdk.internal.vm.compiler_EXCLUDES += \ >> jdk.internal.vm.compiler.collections.test \ >> + org.graalvm.compiler.processor \ >> org.graalvm.compiler.core.match.processor \ >> org.graalvm.compiler.nodeinfo.processor \ >> org.graalvm.compiler.options.processor \ >> org.graalvm.compiler.serviceprovider.processor \ >> - org.graalvm.compiler.replacements.verifier \ >> + org.graalvm.compiler.replacements.processor \ >> + org.graalvm.compiler.replacements.test.jdk9 \ >> org.graalvm.compiler.api.directives.test \ >> org.graalvm.compiler.api.test \ >> org.graalvm.compiler.asm.aarch64.test \ > > This is what the file looks like after I patched it: > > jdk.internal.vm.compiler_EXCLUDES += \ > jdk.internal.vm.compiler.collections.test \ > org.graalvm.compiler.processor \ > org.graalvm.compiler.core.match.processor \ > org.graalvm.compiler.nodeinfo.processor \ > org.graalvm.compiler.options.processor \ > org.graalvm.compiler.serviceprovider.processor \ > org.graalvm.compiler.replacements.processor \ > org.graalvm.compiler.replacements.test.jdk9 \ > org.graalvm.compiler.api.directives.test \ > ... > > so that patch definitely worked. > > But the package is called org.graalvm.compiler.replacements.jdk9.test, not > org.graalvm.compiler.replacements.test.jdk9 Ah yes, it was just recently re-named - sorry for missing that detail. I've updated the patch and attached it (instead of inlining it) to make applying it easier (JBS formatting seems to destroy the applicability of a patch). -Doug > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Tue May 29 16:36:33 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 29 May 2018 17:36:33 +0100 Subject: getAndAddInt() breaks AOT compilation for AArch64 Message-ID: The problem is that the new code assumes that the delta is passed in a register, but the caller does not allocate a register for it. In AArch64AtomicMove.java, delta must be an AllocatableValue, and the caller at AArch64LIRGenerator.java must call asAllocatable(delta). An alternative might be to special-case integer values that will fit in an immediate operand and generate an immediate add. But, please do something: AOT is broken until you patch it. Thank you. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From stewartd at qti.qualcomm.com Tue May 29 16:45:08 2018 From: stewartd at qti.qualcomm.com (Daniel Stewart) Date: Tue, 29 May 2018 16:45:08 +0000 Subject: getAndAddInt() breaks AOT compilation for AArch64 In-Reply-To: References: Message-ID: <9de6d9f5294d47479103091ace7fe110@NASANEXM01E.na.qualcomm.com> Unfortunately I'm on holiday until Friday and have no access to be able to patch this. I'll work on it once I'm back if someone doesn't get to it first. Sorry for the breakage. Daniel ________________________________________ From: Andrew Haley [aph at redhat.com] Sent: Tuesday, May 29, 2018 12:36 PM To: stewartd.qdt Cc: graal-dev at openjdk.java.net Subject: getAndAddInt() breaks AOT compilation for AArch64 The problem is that the new code assumes that the delta is passed in a register, but the caller does not allocate a register for it. In AArch64AtomicMove.java, delta must be an AllocatableValue, and the caller at AArch64LIRGenerator.java must call asAllocatable(delta). An alternative might be to special-case integer values that will fit in an immediate operand and generate an immediate add. But, please do something: AOT is broken until you patch it. Thank you. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Tue May 29 16:57:28 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 29 May 2018 17:57:28 +0100 Subject: getAndAddInt() breaks AOT compilation for AArch64 In-Reply-To: <9de6d9f5294d47479103091ace7fe110@NASANEXM01E.na.qualcomm.com> References: <9de6d9f5294d47479103091ace7fe110@NASANEXM01E.na.qualcomm.com> Message-ID: <95817229-7510-da47-050f-c344dbe530d7@redhat.com> On 05/29/2018 05:45 PM, Daniel Stewart wrote: > Unfortunately I'm on holiday until Friday and have no access to be able to patch this. I'll work on it once I'm back if someone doesn't get to it first. > > Sorry for the breakage. It's OK, we can work around it for now. Better that you get to fix it yourself. And if someone reading this is thinking about importing Graal to OpenJDK, this would not be a good time. :-) -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From dean.long at oracle.com Tue May 29 22:51:39 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 29 May 2018 15:51:39 -0700 Subject: getAndAddInt() breaks AOT compilation for AArch64 In-Reply-To: <95817229-7510-da47-050f-c344dbe530d7@redhat.com> References: <9de6d9f5294d47479103091ace7fe110@NASANEXM01E.na.qualcomm.com> <95817229-7510-da47-050f-c344dbe530d7@redhat.com> Message-ID: <87910b7a-0045-43b1-4e18-8246a53e28c9@oracle.com> On 5/29/18 9:57 AM, Andrew Haley wrote: > On 05/29/2018 05:45 PM, Daniel Stewart wrote: >> Unfortunately I'm on holiday until Friday and have no access to be able to patch this. I'll work on it once I'm back if someone doesn't get to it first. >> >> Sorry for the breakage. > It's OK, we can work around it for now. Better that you > get to fix it yourself. And if someone reading this is thinking > about importing Graal to OpenJDK, this would not be a good time. > :-) > Can we include a work-around in the Graal import rather than delaying it? dl From juan.fumero at manchester.ac.uk Wed May 30 07:11:12 2018 From: juan.fumero at manchester.ac.uk (Juan Fumero) Date: Wed, 30 May 2018 09:11:12 +0200 Subject: Java Local annotations and Graal Inlining In-Reply-To: References: <1527495297.5613.7.camel@manchester.ac.uk> Message-ID: <1527664272.8613.1.camel@manchester.ac.uk> Hi Doug,? ???You are right. I got confused with some of the tooling in Manchester. We will go for parsing with the ASM library. Regards, Juan On Mon, 2018-05-28 at 17:40 +0200, Doug Simon wrote: > Hi Juan, > > I'm a little lost. I don't recall ResolvedJavaMethod ever having a > getLocalAnnotations method. Standard Java reflection doesn't provide > this either. I think you can only get at these by parsing the class > files. > > -Doug > > > On 28 May 2018, at 10:14, Juan Fumero > > wrote: > > > > Hi all,? > > > > I have a question concerning local Java annotations and the > > inlining > > phase in Graal.? > > > > I want to get Java annotations associated with local variables > > (runtime > > annotations). The way we currently do is through the > > ResolvedJavaMethod > > and the getLocalAnnotations method (we based on Graal-0.22). I > > wanted > > to reproduce this example using the last Graal but this method does > > not > > exist anymore.? > > > > Currently, we can see local annotations when we directly compile > > the > > method that contains the annotations. However, when the method > > contains > > a methodInvoke, after inlining we do not see any local annotations > > any > > more. Is this due to an error during inlining?? > > > > To clarify, here I show you a snippet of what we are trying to do.? > > Our goal is to associate a Graal-IR node with a Java annotation. In > > this case, a PhiNode (i var). > > > > > > @Target({ ElementType.LOCAL_VARIABLE, ElementType.TYPE}) > > @Retention(RetentionPolicy.RUNTIME) > > public @interface Local { > > } > > > > > > public void foo(...) { > > ???bar(...) > > } > > > > public void bar(...) { > > ???for (@Local int i = 0, i ... ) { > > > > ???} > > } > > > > > > If I build the Graal-IR for the method `bar`, I see @Local for the > > PhiNode(i var). However, when I compile the method `foo`, I can the > > inlined loop in `foo` after inlining but I do not see @Local > > anymore.? > > The real code is more complicated, `foo` is from an abstract class > > and > > bar is directly the user code. Our compiler has to start compiling > > from > > method `foo`.? > > > > > > Any pointers, suggestions or work around will be appreciated.? > > > > > > Thanks, > > Juan > > > > From zhongwei.yao at linaro.org Wed May 30 09:42:41 2018 From: zhongwei.yao at linaro.org (Zhongwei Yao) Date: Wed, 30 May 2018 17:42:41 +0800 Subject: getAndAddInt() breaks AOT compilation for AArch64 In-Reply-To: <87910b7a-0045-43b1-4e18-8246a53e28c9@oracle.com> References: <9de6d9f5294d47479103091ace7fe110@NASANEXM01E.na.qualcomm.com> <95817229-7510-da47-050f-c344dbe530d7@redhat.com> <87910b7a-0045-43b1-4e18-8246a53e28c9@oracle.com> Message-ID: Hi, all, Since Daniel is on holiday, here are my 2 methods to get it pass: - one is a quick fix for this error (the fix doesn't include optimization for special-case integer values as Andrew mentioned). - and another is a workaround which disables "registerUnsafeReadAndAddPlugins" completely. Here are the patches: http://cr.openjdk.java.net/~zyao/getAndAddInt/ Both of them are OK when running "jaotc --output libjava.base.so --module java.base" and pass Graal unit tests. Could you take a review? And what shall I do if the patch is OK? Shall I create a pull request in Graal for it? Or should we just wait for Daniel's full fix for it? On 30 May 2018 at 06:51, wrote: > On 5/29/18 9:57 AM, Andrew Haley wrote: >> >> On 05/29/2018 05:45 PM, Daniel Stewart wrote: >>> >>> Unfortunately I'm on holiday until Friday and have no access to be able >>> to patch this. I'll work on it once I'm back if someone doesn't get to it >>> first. >>> >>> Sorry for the breakage. >> >> It's OK, we can work around it for now. Better that you >> get to fix it yourself. And if someone reading this is thinking >> about importing Graal to OpenJDK, this would not be a good time. >> :-) >> > > Can we include a work-around in the Graal import rather than delaying it? > > dl -- Best regards, Zhongwei From aph at redhat.com Wed May 30 09:43:51 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 30 May 2018 10:43:51 +0100 Subject: getAndAddInt() breaks AOT compilation for AArch64 In-Reply-To: References: <9de6d9f5294d47479103091ace7fe110@NASANEXM01E.na.qualcomm.com> <95817229-7510-da47-050f-c344dbe530d7@redhat.com> <87910b7a-0045-43b1-4e18-8246a53e28c9@oracle.com> Message-ID: <6a3418bc-5cc7-417e-c57a-7baa6f892eaa@redhat.com> On 05/30/2018 10:42 AM, Zhongwei Yao wrote: > And what shall I do if the patch is OK? Shall I create a pull request > in Graal for it? Or should we just wait for Daniel's full fix for it? Let me do it. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Wed May 30 11:20:09 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 30 May 2018 12:20:09 +0100 Subject: getAndAddInt() breaks AOT compilation for AArch64 In-Reply-To: <6a3418bc-5cc7-417e-c57a-7baa6f892eaa@redhat.com> References: <9de6d9f5294d47479103091ace7fe110@NASANEXM01E.na.qualcomm.com> <95817229-7510-da47-050f-c344dbe530d7@redhat.com> <87910b7a-0045-43b1-4e18-8246a53e28c9@oracle.com> <6a3418bc-5cc7-417e-c57a-7baa6f892eaa@redhat.com> Message-ID: <83162105-4edd-25e4-1ce1-16e8d2200281@redhat.com> On 05/30/2018 10:43 AM, Andrew Haley wrote: > On 05/30/2018 10:42 AM, Zhongwei Yao wrote: >> And what shall I do if the patch is OK? Shall I create a pull request >> in Graal for it? Or should we just wait for Daniel's full fix for it? > > Let me do it. This is all we need for correctness. --- /local/graal/graal/compiler/src/org.graalvm.compiler.core.aarch64/src/org/graalvm/compiler/core/aarch64/AArch64LIRGenerator.java 2018-05-29 11:10:55.653614259 +0100 +++ src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.aarch64/src/org/graalvm/compiler/core/aarch64/AArch64LIRGenerator.java 2018-05-29 17:04:52.622429877 +0100 @@ -152,7 +152,7 @@ Variable scratch1 = newVariable(kind); Variable scratch2 = newVariable(kind); - append(new AtomicReadAndAddOp((AArch64Kind) kind.getPlatformKind(), asAllocatable(result), asAllocatable(address), delta, asAllocatable(scratch1), asAllocatable(scratch2))); + append(new AtomicReadAndAddOp((AArch64Kind) kind.getPlatformKind(), asAllocatable(result), asAllocatable(address), asAllocatable(delta), asAllocatable(scratch1), asAllocatable(scratch2))); return result; } I guess I don't have to create a pull request for this? -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From zhongwei.yao at linaro.org Wed May 30 13:20:21 2018 From: zhongwei.yao at linaro.org (Zhongwei Yao) Date: Wed, 30 May 2018 21:20:21 +0800 Subject: getAndAddInt() breaks AOT compilation for AArch64 In-Reply-To: <6a3418bc-5cc7-417e-c57a-7baa6f892eaa@redhat.com> References: <9de6d9f5294d47479103091ace7fe110@NASANEXM01E.na.qualcomm.com> <95817229-7510-da47-050f-c344dbe530d7@redhat.com> <87910b7a-0045-43b1-4e18-8246a53e28c9@oracle.com> <6a3418bc-5cc7-417e-c57a-7baa6f892eaa@redhat.com> Message-ID: Thanks! On 30 May 2018 at 17:43, Andrew Haley wrote: > On 05/30/2018 10:42 AM, Zhongwei Yao wrote: >> And what shall I do if the patch is OK? Shall I create a pull request >> in Graal for it? Or should we just wait for Daniel's full fix for it? > > Let me do it. > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 -- Best regards, Zhongwei From dean.long at oracle.com Wed May 30 17:10:46 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 30 May 2018 10:10:46 -0700 Subject: getAndAddInt() breaks AOT compilation for AArch64 In-Reply-To: <83162105-4edd-25e4-1ce1-16e8d2200281@redhat.com> References: <9de6d9f5294d47479103091ace7fe110@NASANEXM01E.na.qualcomm.com> <95817229-7510-da47-050f-c344dbe530d7@redhat.com> <87910b7a-0045-43b1-4e18-8246a53e28c9@oracle.com> <6a3418bc-5cc7-417e-c57a-7baa6f892eaa@redhat.com> <83162105-4edd-25e4-1ce1-16e8d2200281@redhat.com> Message-ID: <74169b68-7aa8-13b1-f9ae-13f90f1a75eb@oracle.com> On 5/30/18 4:20 AM, Andrew Haley wrote: > On 05/30/2018 10:43 AM, Andrew Haley wrote: >> On 05/30/2018 10:42 AM, Zhongwei Yao wrote: >>> And what shall I do if the patch is OK? Shall I create a pull request >>> in Graal for it? Or should we just wait for Daniel's full fix for it? >> Let me do it. > This is all we need for correctness. > > --- /local/graal/graal/compiler/src/org.graalvm.compiler.core.aarch64/src/org/graalvm/compiler/core/aarch64/AArch64LIRGenerator.java 2018-05-29 11:10:55.653614259 +0100 > +++ src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.aarch64/src/org/graalvm/compiler/core/aarch64/AArch64LIRGenerator.java 2018-05-29 17:04:52.622429877 +0100 > @@ -152,7 +152,7 @@ > Variable scratch1 = newVariable(kind); > Variable scratch2 = newVariable(kind); > > - append(new AtomicReadAndAddOp((AArch64Kind) kind.getPlatformKind(), asAllocatable(result), asAllocatable(address), delta, asAllocatable(scratch1), asAllocatable(scratch2))); > + append(new AtomicReadAndAddOp((AArch64Kind) kind.getPlatformKind(), asAllocatable(result), asAllocatable(address), asAllocatable(delta), asAllocatable(scratch1), asAllocatable(scratch2))); > return result; > } > > I guess I don't have to create a pull request for this? > I believe a pull request is going to be needed to get it upstream into Graal, but I can include this patch in "JDK-8202670: Update Graal" to allow the update to proceed. dl From doug.simon at oracle.com Wed May 30 17:14:06 2018 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 30 May 2018 19:14:06 +0200 Subject: getAndAddInt() breaks AOT compilation for AArch64 In-Reply-To: <74169b68-7aa8-13b1-f9ae-13f90f1a75eb@oracle.com> References: <9de6d9f5294d47479103091ace7fe110@NASANEXM01E.na.qualcomm.com> <95817229-7510-da47-050f-c344dbe530d7@redhat.com> <87910b7a-0045-43b1-4e18-8246a53e28c9@oracle.com> <6a3418bc-5cc7-417e-c57a-7baa6f892eaa@redhat.com> <83162105-4edd-25e4-1ce1-16e8d2200281@redhat.com> <74169b68-7aa8-13b1-f9ae-13f90f1a75eb@oracle.com> Message-ID: <51828FBC-89A5-4C16-AE57-7ADE93F7BC4C@oracle.com> > On 30 May 2018, at 19:10, dean.long at oracle.com wrote: > > On 5/30/18 4:20 AM, Andrew Haley wrote: >> On 05/30/2018 10:43 AM, Andrew Haley wrote: >>> On 05/30/2018 10:42 AM, Zhongwei Yao wrote: >>>> And what shall I do if the patch is OK? Shall I create a pull request >>>> in Graal for it? Or should we just wait for Daniel's full fix for it? >>> Let me do it. >> This is all we need for correctness. >> >> --- /local/graal/graal/compiler/src/org.graalvm.compiler.core.aarch64/src/org/graalvm/compiler/core/aarch64/AArch64LIRGenerator.java 2018-05-29 11:10:55.653614259 +0100 >> +++ src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.aarch64/src/org/graalvm/compiler/core/aarch64/AArch64LIRGenerator.java 2018-05-29 17:04:52.622429877 +0100 >> @@ -152,7 +152,7 @@ >> Variable scratch1 = newVariable(kind); >> Variable scratch2 = newVariable(kind); >> >> - append(new AtomicReadAndAddOp((AArch64Kind) kind.getPlatformKind(), asAllocatable(result), asAllocatable(address), delta, asAllocatable(scratch1), asAllocatable(scratch2))); >> + append(new AtomicReadAndAddOp((AArch64Kind) kind.getPlatformKind(), asAllocatable(result), asAllocatable(address), asAllocatable(delta), asAllocatable(scratch1), asAllocatable(scratch2))); >> return result; >> } >> >> I guess I don't have to create a pull request for this? >> > > I believe a pull request is going to be needed to get it upstream into Graal, but I can include this patch in "JDK-8202670: Update Graal" to allow the update to proceed. Yes, you should manually include it in "JDK-8202670: Update Graal". I assume a more complete PR will later be submitted by Daniel to github. Or is this already the final fix Andrew? -Doug From aph at redhat.com Wed May 30 17:46:41 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 30 May 2018 18:46:41 +0100 Subject: getAndAddInt() breaks AOT compilation for AArch64 In-Reply-To: <51828FBC-89A5-4C16-AE57-7ADE93F7BC4C@oracle.com> References: <9de6d9f5294d47479103091ace7fe110@NASANEXM01E.na.qualcomm.com> <95817229-7510-da47-050f-c344dbe530d7@redhat.com> <87910b7a-0045-43b1-4e18-8246a53e28c9@oracle.com> <6a3418bc-5cc7-417e-c57a-7baa6f892eaa@redhat.com> <83162105-4edd-25e4-1ce1-16e8d2200281@redhat.com> <74169b68-7aa8-13b1-f9ae-13f90f1a75eb@oracle.com> <51828FBC-89A5-4C16-AE57-7ADE93F7BC4C@oracle.com> Message-ID: <8614a86c-747f-8bd7-b952-b504566d2bf1@redhat.com> On 05/30/2018 06:14 PM, Doug Simon wrote: > Yes, you should manually include it in "JDK-8202670: Update Graal". I assume a more complete PR will later be submitted by Daniel to github. Or is this already the final fix Andrew? I think we can do better, so let's just have this for OpenJDK. Thanks. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From vladimir.kozlov at oracle.com Thu May 31 15:03:54 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 31 May 2018 08:03:54 -0700 Subject: RFR(XL) 8202670: Update Graal In-Reply-To: <67ef78f6-9584-82c4-d312-f1e5df328995@oracle.com> References: <63c90018-2b08-41e0-fc76-f390580cf59b@oracle.com> <67ef78f6-9584-82c4-d312-f1e5df328995@oracle.com> Message-ID: <6212fd70-92a8-8e23-c18d-a37f0a455426@oracle.com> Looks good. thanks, Vladimir On 5/30/18 12:44 PM, dean.long at oracle.com wrote: > Here's the patch to fix aarch64: > > ??? http://cr.openjdk.java.net/~dlong/8202670/aarch64-patch/ > > from the following thread: > > http://mail.openjdk.java.net/pipermail/graal-dev/2018-May/005394.html > > dl > > On 5/29/18 2:51 PM, dean.long at oracle.com wrote: >> I just saw the "getAndAddInt() breaks AOT compilation for AArch64" >> thread: >> >> "And if someone reading this is thinking >> >> about importing Graal to OpenJDK, this would not be a good time. >> " >> >> so I guess I should delay this a bit? >> >> dl >> >> On 5/29/18 2:47 PM, dean.long at oracle.com wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8202670 >>> http://cr.openjdk.java.net/~dlong/8202670/ {graal,patch} >>> >>> See the bug for the full list of Graal changes. >>> >>> Andrew Haley, please check that these changes are still working: >>> >>> ?? [GR-9604] AArch64: Integrate AOT Implementation. >>> >>> dl >> > From dean.long at oracle.com Thu May 31 17:34:47 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 31 May 2018 10:34:47 -0700 Subject: RFR(XL) 8202670: Update Graal In-Reply-To: <6212fd70-92a8-8e23-c18d-a37f0a455426@oracle.com> References: <63c90018-2b08-41e0-fc76-f390580cf59b@oracle.com> <67ef78f6-9584-82c4-d312-f1e5df328995@oracle.com> <6212fd70-92a8-8e23-c18d-a37f0a455426@oracle.com> Message-ID: <9a5b6e81-2de5-0c79-1a1c-7040ec0576a0@oracle.com> Thanks Vladimir and Andrew. dl On 5/31/18 8:03 AM, Vladimir Kozlov wrote: > Looks good. > > thanks, > Vladimir > > On 5/30/18 12:44 PM, dean.long at oracle.com wrote: >> Here's the patch to fix aarch64: >> >> ???? http://cr.openjdk.java.net/~dlong/8202670/aarch64-patch/ >> >> from the following thread: >> >> http://mail.openjdk.java.net/pipermail/graal-dev/2018-May/005394.html >> >> dl >> >> On 5/29/18 2:51 PM, dean.long at oracle.com wrote: >>> I just saw the "getAndAddInt() breaks AOT compilation for AArch64" >>> thread: >>> >>> "And if someone reading this is thinking >>> >>> about importing Graal to OpenJDK, this would not be a good time. >>> " >>> >>> so I guess I should delay this a bit? >>> >>> dl >>> >>> On 5/29/18 2:47 PM, dean.long at oracle.com wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8202670 >>>> http://cr.openjdk.java.net/~dlong/8202670/ {graal,patch} >>>> >>>> See the bug for the full list of Graal changes. >>>> >>>> Andrew Haley, please check that these changes are still working: >>>> >>>> ?? [GR-9604] AArch64: Integrate AOT Implementation. >>>> >>>> dl >>> >>