BackEnd Service Provider.
Gary Frost
frost.gary at gmail.com
Tue Apr 21 11:23:58 UTC 2020
Here is a public github project
https://github.com/grfrost/graal_service_issue
git clone https://github.com/grfrost/graal_service_issue.git
cd graal_service_issue
# vi Makefile and set path to your JDK13 (JDK11 may work... I know some
API's have changed between 11 and 13)
make
Gary
On Tue, Apr 21, 2020 at 11:59 AM Gary Frost <frost.gary at gmail.com> wrote:
> I apologise Doug
>
> My argument is that there is no combination of '-add-exports and
> --add-reads options to javac and java' that will work here. Try it! ;)
>
> I do/did in fact have it at runtime as well. I stripped it from my
> description because it seems to make no difference.
>
> /usr/lib/jvm/jdk-13/bin/java \
> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler
> -XX:-TieredCompilation\
>
> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\
>
> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\
> --add-exports
> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\
> --add-exports
> jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\
> --add-exports
> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\
> -p build -m org.grfstuff.compiler\
> -classpath build/org.grfstuff.violajones\
> org.grfstuff.violajones.ViolaJones
>
> I attached a zip of the project to my last email and it got stripped.
>
> My argument is that there is no combination of '-add-exports and
> --add-reads options to javac and java' that will work.
>
> The only option is --patch-module which is not even officially offered by
> javac or java and is not supported by IDE's
>
> $ /usr/lib/jvm/jdk-13/bin/javac --help | grep patch | wc -l
> 0
> $ /usr/lib/jvm/jdk-13/bin/java --help | grep patch | wc -l
> 0
>
> I only know of this option because I read the source code ;)
>
> Gary
>
> On Tue, Apr 21, 2020 at 11:47 AM Doug Simon <doug.simon at oracle.com> wrote:
>
>>
>>
>> On 21 Apr 2020, at 12:30, Gary Frost <frost.gary at gmail.com> wrote:
>>
>> I don't want to flog this horse excessively, but please humour me one
>> more time.
>>
>> Even after defining a module I can't offer my own service provider for
>> org.graalvm.compiler.hotspot.HotSpotBackendFactory
>>
>> I really think we need a way for folks to experiment with Graal backends
>> without having to build there own JDK/JVM.
>>
>> I created a module to be able to map my service provider
>>
>> src/org.grfstuff.compiler/
>> ├── module-info.java
>> └── org
>> └── grfstuff
>> └── compiler
>> └── GrfStuffHotSpotBackendFactory.java
>>
>>
>> Where my module-info.java is this.
>> --8<--
>> module org.grfstuff.compiler {
>> requires jdk.internal.vm.compiler;
>> provides org.graalvm.compiler.hotspot.HotSpotBackendFactory with
>> org.grfstuff.compiler.GrfStuffHotSpotBackendFactory;
>> }
>> -->8--
>>
>> My implementation is just enough code so I can see an instance of my
>> backend get loaded.
>> --8<--
>> package org.grfstuff.compiler;
>> import org.graalvm.compiler.hotspot.HotSpotBackendFactory;
>> import org.graalvm.compiler.serviceprovider.ServiceProvider;
>> import org.graalvm.compiler.hotspot.amd64.AMD64HotSpotBackendFactory;
>>
>> @ServiceProvider(HotSpotBackendFactory.class)
>> public class GrfStuffHotSpotBackendFactory extends
>> AMD64HotSpotBackendFactory {
>> public GrfStuffHotSpotBackendFactory(){
>> System.out.println("GrfStuffHotSpotFactory");
>> }
>> public String toString() {
>> return "GRFSTUFF_AMD64";
>> }
>> }
>> -->8--
>>
>> I can build my module using appropriate --add-exports 'jiggery pokery ' :)
>>
>> javac -g \
>> -d build \
>> --add-exports
>> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\
>> --add-exports
>> jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\
>> --add-exports
>> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\
>> --module-source-path src\
>> --module org.grfstuff.compiler\
>> $(shell find src/org.grfstuff.compiler -name *.java)
>>
>> But at runtime (my app is a compute intensive voilajones face detection
>> algorithm)
>> java \
>> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler
>> -XX:-TieredCompilation\
>>
>> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\
>>
>> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\
>> -p build -m org.grfstuff.compiler\
>> -classpath build/org.grfstuff.violajones\
>> org.grfstuff.violajones.ViolaJones
>>
>> I get this error
>>
>> Error occurred during initialization of boot layer
>> java.lang.module.ResolutionException: Module org.grfstuff.compiler does
>> not read a module that exports org.graalvm.compiler.hotspot
>>
>> It would appear that whilst add-exports is enough at compile time to
>> persuade the compiler that all is well. At runtime the service loader
>> execution path does not honour the add-exports requests to the service
>> layer.
>>
>>
>> If you use —add-exports during java compilation, you almost certainly
>> need to use it at runtime. That is, repeat your --add-export arguments to
>> the java launcher. You may also need some --add-reads options. If you put
>> you jars up somewhere, I can help find the right incantation if you’re
>> still having trouble.
>>
>> In the end my temporary solution was to add this stanza
>> to org.graalvm.compiler.hotspot.CompilerConfigurationFactory.DefaultBackendMap's
>> constructor.
>>
>> --8<--
>> String backendFactoryClassName =
>> System.getProperty("graal.backend.factory"); //
>> org.grfstuff.compiler.GrfStuffHotSpotBackendFactory
>> if (backendFactoryClassName != null){
>> try {
>> Class<? extends HotSpotBackendFactory> factoryClass
>> = (Class<? extends HotSpotBackendFactory>)
>> Class.forName(backendFactoryClassName);
>> HotSpotBackendFactory factory =
>> factoryClass.getDeclaredConstructor().newInstance();
>> Class<? extends Architecture> arch =
>> factory.getArchitecture();
>> HotSpotBackendFactory oldEntry =
>> backends.put(arch, factory);
>> }catch(Exception e){
>>
>> }
>> }else {
>> //existing code
>> }
>> -->8--
>>
>> Then I can build a jdk13 image and run using a combination of
>> --add-modules along with
>> -Dgraal.backend.factory=org.grfstuff.compiler.GrfStuffHotSpotBackendFactory
>>
>> Whilst this works for me. I am quite sure if our common goal is to get
>> users to 'adopt/play-with' Graal, we can't expect them to build a dedicated
>> OpenJDK image.
>>
>> I think it is an oversite to offer a ServiceProvider extension mechanism
>> (HotSpotBackendFactory) only for the module system to forbid anyone from
>> using it.
>>
>>
>> This is a constraint imposed on us by the fact that Graal is part of the
>> JDK. Just like Unsafe, JVMCI and Graal cannot be publicly exported. I think
>> the inconvenience of adding a few --add-exports and --add-reads options to
>> javac and java is an acceptable overhead to experiment with Graal. We just
>> need to get the incantation right and hope someone writes a nice blog
>> article describing it ;-)
>>
>> It would be great if we could offer a simpler mechanism (without
>> resorting to --patch-module which does not seem to be supported by
>> IntelliJ).
>>
>> We already have some protection by forcing
>> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler maybe we can just
>> open this up a little more behind these current flags.
>>
>>
>> As stated above, the JVMCI and Graal modules cannot be opened up by
>> default.
>>
>> -Doug
>>
>>
>>
>> On Mon, Apr 20, 2020 at 1:52 PM Doug Simon <doug.simon at oracle.com> wrote:
>>
>>> Hi Gary,
>>>
>>> On 20 Apr 2020, at 14:44, Gary Frost <frost.gary at gmail.com> wrote:
>>>
>>> Yeah I started created a module, but it seems for my module to compile I
>>> need jdk.internal.vm.ci
>>> <https://urldefense.com/v3/__http://jdk.internal.vm.ci__;!!GqivPVa7Brio!IQVX1iggkDspLlkVBXNQ7IulcpTkED_VhK6SjF2apyR4-XW-E7oZHqeQPPY4GJB0$>
>>> module to export org.graalvm.compiler.hotspot.HotSpotBackendFactory to
>>> my named module. Which seems wrong.
>>>
>>>
>>> Actually, that’s by design. Only standard JDK API can be exported
>>> unqualified from the JDK. Neither Graal nor JVMCI is public API and so
>>> --add-exports is needed for compilation.
>>>
>>>
>>> I wanted to avoid using GraalVM. I figured I should be able to do what
>>> I want, using stock/released JDK13 and -XX:+UnlockExperimentalVMOptions
>>> -XX:+UseJVMCICompiler
>>>
>>>
>>> You can - you just need --add-exports (and maybe some other options) to
>>> “open up” the classes you’re trying to hook into.
>>>
>>> I can 'get around it' by pulling the source for
>>> org.graalvm.compiler.hotspot.CompilerConfigurationFactory locally. Hacking
>>> this one file (to load my Factory) then building using
>>> using --patch-module
>>> jdk.internal.vm.compiler=src/jdk.internal.vm.compiler/java, but this is
>>> clearly hacky. I should not have to use --patch-module here.
>>>
>>> Alas using --patch-module renders IntelliJ next to useless (it ignores
>>> patch-module) as pointed out by John Rose and myself here.
>>> https://youtrack.jetbrains.com/issue/IDEA-224342
>>> <https://urldefense.com/v3/__https://youtrack.jetbrains.com/issue/IDEA-224342__;!!GqivPVa7Brio!IQVX1iggkDspLlkVBXNQ7IulcpTkED_VhK6SjF2apyR4-XW-E7oZHqeQPKLKl6Nu$>
>>>
>>> Maybe the intent was that the JDK versions of Graal, were just to 'use
>>> Graal', not to allow developers to add there own features.
>>>
>>> Maybe that is the distiction between GraalVM and Graal enabled JDK.
>>>
>>>
>>> No, both of these don’t not offer open extension points. You need to
>>> force it open on the command as mentioned above. We can’t have just any old
>>> app code getting their hands on the compiler ;-)
>>>
>>> -Doug
>>>
>>> On Mon, Apr 20, 2020 at 11:54 AM Doug Simon <doug.simon at oracle.com>
>>> wrote:
>>>
>>>> Hi Gary,
>>>>
>>>> Have you tried creating a module for your service? I think that’s
>>>> required as of JDK 9.
>>>>
>>>> I suggest opening a PR on https://github.com/oracle/graal
>>>> <https://urldefense.com/v3/__https://github.com/oracle/graal__;!!GqivPVa7Brio!IQVX1iggkDspLlkVBXNQ7IulcpTkED_VhK6SjF2apyR4-XW-E7oZHqeQPMQSQQsk$>
>>>> so we can have a more concrete discussion.
>>>>
>>>> -Doug
>>>>
>>>> > On 20 Apr 2020, at 11:17, Gary Frost <frost.gary at gmail.com> wrote:
>>>> >
>>>> > I feel I am missing something obvious ;)
>>>> >
>>>> > I am working (well playing actually) with Graal and JDK13 and have
>>>> built my
>>>> > own implementation of
>>>> org.graalvm.compiler.hotspot.HotSpotBackendFactory to
>>>> > inject some custom code into generated x86. Very cool.
>>>> >
>>>> > If I hack the
>>>> org.graalvm.compiler.hotspot.CompilerConfigurationFactory
>>>> > class (from JDK13 source) I can instantiate my factory directly, and
>>>> I see
>>>> > my modified code.
>>>> >
>>>> > But of course I want to able to build against and use use my factory
>>>> using
>>>> > a clean opendjdk JDK13 binary distribution.
>>>> >
>>>> > It looks like the
>>>> > class org.graalvm.compiler.hotspot.CompilerConfigurationFactory is
>>>> setup to
>>>> > allow me to offer my factory as a 'service provider' which the
>>>> > CompilerConfigurationFactory accesses via this call
>>>> > GraalServices.load(HotSpotBackendFactory.class)
>>>> > but nothing I try works.
>>>> >
>>>> > It looks like I need to provide a module-info.java class, but of
>>>> course I
>>>> > only want my one service to override the default.
>>>> >
>>>> > Any suggestions/help oir pointers to docs would be appreciated.
>>>> >
>>>> > Gary
>>>>
>>>>
>>> <graal_service_issue.zip>
>>
>>
>>
More information about the graal-dev
mailing list