building a JDK that includes the hsail graal backend without building graal

Thu Jul 10 14:28:59 UTC 2014

Yes, things worked when I disabled assertions.  Thanks.

-- Tom

-----Original Message-----
From: graal-dev [mailto:graal-dev-bounces at openjdk.java.net] On Behalf Of Christian Wimmer
Sent: Wednesday, July 09, 2014 6:15 PM
To: graal-dev at openjdk.java.net
Subject: Re: building a JDK that includes the hsail graal backend without building graal

The variable names are just used for assertion checking, so no one must 
rely on that they are non-null.

When you disable assertions in Graal, your NullPointerException should 
go away too because SnippetInfo$Lazy.initNames is only called from 
within an assert statement.

-Christian

On 07/09/2014 04:05 PM, Tom Rodriguez wrote:
> That code is assuming it's always compiled -g or at least -g:vars, which is true when built with mx.  Only initializing the name when it's non-null should make it work, assuming no one else it relying on that being non-null.
>
> tom
>
> On Jul 9, 2014, at 3:43 PM, Deneau, Tom <tom.deneau at amd.com> wrote:
>
>> At the request of some of the Hotspot team, I am trying an experiment to do the following:
>>
>> As you are all probably aware, currently our build process to build a Sumatra JDK image is two steps:
>>
>>
>> 1)  Build the Sumatra-modified JDK
>>
>> a.  there are a small number of classes (less than 10) from java.util.stream that get modified to enable the gpu offload.  If offload is detected it then calls into graal.
>>
>>
>>
>>   2) With JAVA_HOME set to the Sumatra-modified JDK, build graal "server" compiler.
>>
>>      This uses its own special "mx build" command.
>>
>>      This produces a new JDK image with the following changes:
>>
>>         2.1) from the graal java sources, a graal.jar in the jre/lib directory
>>
>>         2.2) from the graal hotspot sources, a new hotspot libjvm.so in the jre/lib/amd64/server directory
>>
>>
>>
>>
>> We wanted to add enough classes to sumatra-dev/jdk and sumatra-dev/hotspot so that we could skip step 2 above but still build a functional sumatra able to offload to hsail.  So the equivalent of graal.jar would be built into rt.jar ideally includes only that part of graal java and graal hotspot that would be necessary for an HSAIL backend.   A webrev from before vs. after then would let use see what the hsail/graal additional footprint would be. But for a first cut, we wouldn't spend too much time getting down to the minimum footprint.
>>
>> So far we have some success.
>>
>> I modified mx/projects so that what I thought were the unnecessary project sets would not be included in graal.jar (so truffle, sparc, ptx.  Note that we do need the graal amd64 backend because of the way hsail deoptimization is done).
>>
>> I did our normal 2-part sumatra build and everything seemed to work, and if you look at graal.jar you can see it does not have the classes that were removed via mx projects.
>>
>> Then I took the graal java sources for the classes that were still in graal.jar and added them to the sumatra-dev/jdk.  I also took the hotspot src tree from graal (which is a modification of hs25-b63) and used that instead of the vanilla hs25-b63 tag in sumatra-dev/hotspot.
>>
>> After a few hiccups like having to manually add the graal-generated files graalRuntime.inline.hpp and HotspotVMConfig.inline.hpp to the sumatra-dev/hotspot, things built.
>>
>> And I am able to run sumatra offloads to HSAIL as long as I turn off HSAILDeoptimization.
>>
>> But with HSAILDeoptimization on (the default), I see a NPE coming from
>>
>>
>> com.oracle.graal.graph.GraalGraphInternalError: com.oracle.graal.compiler.common.GraalInternalError: java.lang.NullPointerException
>>
>>        at lir instruction: B2 at 38 DEOPT_CALLER stack:16|j
>>
>> [B0, B2, B1]
>>
>>        at lir instruction: B2 at 38 DEOPT_CALLER stack:16|j
>>
>> [B0, B2, B1]
>>
>>        at node: 12|VMError
>>
>>        at com.oracle.graal.graph.GraalGraphInternalError.transformAndAddContext(GraalGraphInternalError.java:136)
>>
>>        at com.oracle.graal.compiler.gen.NodeLIRBuilder.doBlock(NodeLIRBuilder.java:220)
>>
>>        at com.oracle.graal.compiler.GraalCompiler.emitBlock(GraalCompiler.java:216)
>>
>>        at com.oracle.graal.compiler.GraalCompiler.emitLIR(GraalCompiler.java:250)
>>
>>        at com.oracle.graal.compiler.GraalCompiler.emitBackEnd(GraalCompiler.java:198)
>>
>>        at com.oracle.graal.compiler.GraalCompiler.compileGraph(GraalCompiler.java:141)
>>
>>        at com.oracle.graal.hotspot.hsail.HSAILHotSpotBackend.installKernel(HSAILHotSpotBackend.java:286)
>>
>>        at com.oracle.graal.hotspot.hsail.ForEachToGraal.getCompiledLambda(ForEachToGraal.java:143)
>>
>>        at com.oracle.graal.hotspot.hsail.ForEachToGraal.createKernel(ForEachToGraal.java:149)
>>
>>        at java.util.stream.PipelineInfo.offloadForEachPipeline(PipelineInfo.java:391)
>>
>>
>>
>> Caused by: java.lang.NullPointerException
>>
>>        at com.oracle.graal.replacements.SnippetTemplate$SnippetInfo$Lazy.initNames(SnippetTemplate.java:118)
>>
>>        at com.oracle.graal.replacements.SnippetTemplate$SnippetInfo$Lazy.<init>(SnippetTemplate.java:102)
>>
>>        at com.oracle.graal.replacements.SnippetTemplate$SnippetInfo.lazy(SnippetTemplate.java:146)
>>
>>        at com.oracle.graal.replacements.SnippetTemplate$SnippetInfo.getParameterCount(SnippetTemplate.java:174)
>>
>>        at com.oracle.graal.replacements.SnippetTemplate$CacheKey.<init>(SnippetTemplate.java:384)
>>
>>        at com.oracle.graal.replacements.SnippetTemplate$Arguments.<init>(SnippetTemplate.java:221)
>>
>>        at com.oracle.graal.hotspot.stubs.SnippetStub.makeArguments(SnippetStub.java:99)
>>
>>        at com.oracle.graal.hotspot.stubs.SnippetStub.getGraph(SnippetStub.java:92)
>>
>>        at com.oracle.graal.hotspot.stubs.Stub.getCode(Stub.java:148)
>>
>>
>>
>> so we are in the code that builds the trampoline host code (which explains why this only shows up in the hsail deoptimization codepath).
>>
>>
>>
>> When I step into SnippetInfo$Lazy.initNames, I can see that the NPE is coming because method.getLocalVariableTable() is returning null because   in HotSpotResolvedMethod.getLocalVariableTable() we see that (getConstMethodFlags() & runtime().getConfig().constMethodHasLocalVariableTable) == 0);
>>
>>
>>
>>            private boolean initNames(ResolvedJavaMethod method, int parameterCount) {
>>
>>                 names = new String[parameterCount];
>>
>>                 int slotIdx = 0;
>>
>>                 for (int i = 0; i < names.length; i++) {
>>
>>                     names[i] = method.getLocalVariableTable().getLocal(slotIdx, 0).getName();
>>
>>
>>
>>                     Kind kind = method.getSignature().getParameterKind(i);
>>
>>                     slotIdx += kind == Kind.Long || kind == Kind.Double ? 2 : 1;
>>
>>                 }
>>
>>                 return true;
>>
>>             }
>>
>> any suggestions?
>>
>> I had assumed that the HotSpotVMConfig.inline.hpp that was produced in the graal build could just be used as is in sumatra-dev/hotspot but I don't really understand whether that is a valid assumption.
>>
>> -- Tom
>