building a JDK that includes the hsail graal backend without building graal

[Deneau, Tom]

At the request of some of the Hotspot team, I am trying an experiment to do the following:

As you are all probably aware, currently our build process to build a Sumatra JDK image is two steps:


1)  Build the Sumatra-modified JDK

a.  there are a small number of classes (less than 10) from java.util.stream that get modified to enable the gpu offload.  If offload is detected it then calls into graal.



  2) With JAVA_HOME set to the Sumatra-modified JDK, build graal "server" compiler.

     This uses its own special "mx build" command.

     This produces a new JDK image with the following changes:

        2.1) from the graal java sources, a graal.jar in the jre/lib directory

        2.2) from the graal hotspot sources, a new hotspot libjvm.so in the jre/lib/amd64/server directory




We wanted to add enough classes to sumatra-dev/jdk and sumatra-dev/hotspot so that we could skip step 2 above but still build a functional sumatra able to offload to hsail.  So the equivalent of graal.jar would be built into rt.jar ideally includes only that part of graal java and graal hotspot that would be necessary for an HSAIL backend.   A webrev from before vs. after then would let use see what the hsail/graal additional footprint would be. But for a first cut, we wouldn't spend too much time getting down to the minimum footprint.

So far we have some success.

I modified mx/projects so that what I thought were the unnecessary project sets would not be included in graal.jar (so truffle, sparc, ptx.  Note that we do need the graal amd64 backend because of the way hsail deoptimization is done).

I did our normal 2-part sumatra build and everything seemed to work, and if you look at graal.jar you can see it does not have the classes that were removed via mx projects.

Then I took the graal java sources for the classes that were still in graal.jar and added them to the sumatra-dev/jdk.  I also took the hotspot src tree from graal (which is a modification of hs25-b63) and used that instead of the vanilla hs25-b63 tag in sumatra-dev/hotspot.

After a few hiccups like having to manually add the graal-generated files graalRuntime.inline.hpp and HotspotVMConfig.inline.hpp to the sumatra-dev/hotspot, things built.

And I am able to run sumatra offloads to HSAIL as long as I turn off HSAILDeoptimization.

But with HSAILDeoptimization on (the default), I see a NPE coming from


com.oracle.graal.graph.GraalGraphInternalError: com.oracle.graal.compiler.common.GraalInternalError: java.lang.NullPointerException

       at lir instruction: B2 at 38 DEOPT_CALLER stack:16|j

[B0, B2, B1]

       at lir instruction: B2 at 38 DEOPT_CALLER stack:16|j

[B0, B2, B1]

       at node: 12|VMError

       at com.oracle.graal.graph.GraalGraphInternalError.transformAndAddContext(GraalGraphInternalError.java:136)

       at com.oracle.graal.compiler.gen.NodeLIRBuilder.doBlock(NodeLIRBuilder.java:220)

       at com.oracle.graal.compiler.GraalCompiler.emitBlock(GraalCompiler.java:216)

       at com.oracle.graal.compiler.GraalCompiler.emitLIR(GraalCompiler.java:250)

       at com.oracle.graal.compiler.GraalCompiler.emitBackEnd(GraalCompiler.java:198)

       at com.oracle.graal.compiler.GraalCompiler.compileGraph(GraalCompiler.java:141)

       at com.oracle.graal.hotspot.hsail.HSAILHotSpotBackend.installKernel(HSAILHotSpotBackend.java:286)

       at com.oracle.graal.hotspot.hsail.ForEachToGraal.getCompiledLambda(ForEachToGraal.java:143)

       at com.oracle.graal.hotspot.hsail.ForEachToGraal.createKernel(ForEachToGraal.java:149)

       at java.util.stream.PipelineInfo.offloadForEachPipeline(PipelineInfo.java:391)



Caused by: java.lang.NullPointerException

       at com.oracle.graal.replacements.SnippetTemplate$SnippetInfo$Lazy.initNames(SnippetTemplate.java:118)

       at com.oracle.graal.replacements.SnippetTemplate$SnippetInfo$Lazy.<init>(SnippetTemplate.java:102)

       at com.oracle.graal.replacements.SnippetTemplate$SnippetInfo.lazy(SnippetTemplate.java:146)

       at com.oracle.graal.replacements.SnippetTemplate$SnippetInfo.getParameterCount(SnippetTemplate.java:174)

       at com.oracle.graal.replacements.SnippetTemplate$CacheKey.<init>(SnippetTemplate.java:384)

       at com.oracle.graal.replacements.SnippetTemplate$Arguments.<init>(SnippetTemplate.java:221)

       at com.oracle.graal.hotspot.stubs.SnippetStub.makeArguments(SnippetStub.java:99)

       at com.oracle.graal.hotspot.stubs.SnippetStub.getGraph(SnippetStub.java:92)

       at com.oracle.graal.hotspot.stubs.Stub.getCode(Stub.java:148)



so we are in the code that builds the trampoline host code (which explains why this only shows up in the hsail deoptimization codepath).



When I step into SnippetInfo$Lazy.initNames, I can see that the NPE is coming because method.getLocalVariableTable() is returning null because   in HotSpotResolvedMethod.getLocalVariableTable() we see that (getConstMethodFlags() & runtime().getConfig().constMethodHasLocalVariableTable) == 0);



           private boolean initNames(ResolvedJavaMethod method, int parameterCount) {

                names = new String[parameterCount];

                int slotIdx = 0;

                for (int i = 0; i < names.length; i++) {

                    names[i] = method.getLocalVariableTable().getLocal(slotIdx, 0).getName();



                    Kind kind = method.getSignature().getParameterKind(i);

                    slotIdx += kind == Kind.Long || kind == Kind.Double ? 2 : 1;

                }

                return true;

            }

any suggestions?

I had assumed that the HotSpotVMConfig.inline.hpp that was produced in the graal build could just be used as is in sumatra-dev/hotspot but I don't really understand whether that is a valid assumption.

-- Tom