BackEnd Service Provider.
Gary Frost
frost.gary at gmail.com
Tue Apr 21 12:37:13 UTC 2020
I also added a makefile which uses patch-module which works.
So
git clone https://github.com/grfrost/graal_service_issue.git
cd graal_service_issue
# vi makefile* and set path to your JDK13 (JDK11 may work... I know some
API's have changed between 11 and 13)
make -f makefile-using-module-service-provider
make -f makefile-with-patch-module
On Tue, Apr 21, 2020 at 12:23 PM Gary Frost <frost.gary at gmail.com> wrote:
> Here is a public github project
> https://github.com/grfrost/graal_service_issue
>
> git clone https://github.com/grfrost/graal_service_issue.git
> cd graal_service_issue
> # vi Makefile and set path to your JDK13 (JDK11 may work... I know some
> API's have changed between 11 and 13)
> make
>
> Gary
>
>
>
>
> On Tue, Apr 21, 2020 at 11:59 AM Gary Frost <frost.gary at gmail.com> wrote:
>
>> I apologise Doug
>>
>> My argument is that there is no combination of '-add-exports and
>> --add-reads options to javac and java' that will work here. Try it! ;)
>>
>> I do/did in fact have it at runtime as well. I stripped it from my
>> description because it seems to make no difference.
>>
>> /usr/lib/jvm/jdk-13/bin/java \
>> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler
>> -XX:-TieredCompilation\
>>
>> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\
>>
>> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\
>> --add-exports
>> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\
>> --add-exports
>> jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\
>> --add-exports
>> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\
>> -p build -m org.grfstuff.compiler\
>> -classpath build/org.grfstuff.violajones\
>> org.grfstuff.violajones.ViolaJones
>>
>> I attached a zip of the project to my last email and it got stripped.
>>
>> My argument is that there is no combination of '-add-exports and
>> --add-reads options to javac and java' that will work.
>>
>> The only option is --patch-module which is not even officially offered by
>> javac or java and is not supported by IDE's
>>
>> $ /usr/lib/jvm/jdk-13/bin/javac --help | grep patch | wc -l
>> 0
>> $ /usr/lib/jvm/jdk-13/bin/java --help | grep patch | wc -l
>> 0
>>
>> I only know of this option because I read the source code ;)
>>
>> Gary
>>
>> On Tue, Apr 21, 2020 at 11:47 AM Doug Simon <doug.simon at oracle.com>
>> wrote:
>>
>>>
>>>
>>> On 21 Apr 2020, at 12:30, Gary Frost <frost.gary at gmail.com> wrote:
>>>
>>> I don't want to flog this horse excessively, but please humour me one
>>> more time.
>>>
>>> Even after defining a module I can't offer my own service provider for
>>> org.graalvm.compiler.hotspot.HotSpotBackendFactory
>>>
>>> I really think we need a way for folks to experiment with Graal backends
>>> without having to build there own JDK/JVM.
>>>
>>> I created a module to be able to map my service provider
>>>
>>> src/org.grfstuff.compiler/
>>> ├── module-info.java
>>> └── org
>>> └── grfstuff
>>> └── compiler
>>> └── GrfStuffHotSpotBackendFactory.java
>>>
>>>
>>> Where my module-info.java is this.
>>> --8<--
>>> module org.grfstuff.compiler {
>>> requires jdk.internal.vm.compiler;
>>> provides org.graalvm.compiler.hotspot.HotSpotBackendFactory with
>>> org.grfstuff.compiler.GrfStuffHotSpotBackendFactory;
>>> }
>>> -->8--
>>>
>>> My implementation is just enough code so I can see an instance of my
>>> backend get loaded.
>>> --8<--
>>> package org.grfstuff.compiler;
>>> import org.graalvm.compiler.hotspot.HotSpotBackendFactory;
>>> import org.graalvm.compiler.serviceprovider.ServiceProvider;
>>> import org.graalvm.compiler.hotspot.amd64.AMD64HotSpotBackendFactory;
>>>
>>> @ServiceProvider(HotSpotBackendFactory.class)
>>> public class GrfStuffHotSpotBackendFactory extends
>>> AMD64HotSpotBackendFactory {
>>> public GrfStuffHotSpotBackendFactory(){
>>> System.out.println("GrfStuffHotSpotFactory");
>>> }
>>> public String toString() {
>>> return "GRFSTUFF_AMD64";
>>> }
>>> }
>>> -->8--
>>>
>>> I can build my module using appropriate --add-exports 'jiggery pokery '
>>> :)
>>>
>>> javac -g \
>>> -d build \
>>> --add-exports
>>> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\
>>> --add-exports
>>> jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\
>>> --add-exports
>>> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\
>>> --module-source-path src\
>>> --module org.grfstuff.compiler\
>>> $(shell find src/org.grfstuff.compiler -name *.java)
>>>
>>> But at runtime (my app is a compute intensive voilajones face detection
>>> algorithm)
>>> java \
>>> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler
>>> -XX:-TieredCompilation\
>>>
>>> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\
>>>
>>> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\
>>> -p build -m org.grfstuff.compiler\
>>> -classpath build/org.grfstuff.violajones\
>>> org.grfstuff.violajones.ViolaJones
>>>
>>> I get this error
>>>
>>> Error occurred during initialization of boot layer
>>> java.lang.module.ResolutionException: Module org.grfstuff.compiler does
>>> not read a module that exports org.graalvm.compiler.hotspot
>>>
>>> It would appear that whilst add-exports is enough at compile time to
>>> persuade the compiler that all is well. At runtime the service loader
>>> execution path does not honour the add-exports requests to the service
>>> layer.
>>>
>>>
>>> If you use —add-exports during java compilation, you almost certainly
>>> need to use it at runtime. That is, repeat your --add-export arguments to
>>> the java launcher. You may also need some --add-reads options. If you put
>>> you jars up somewhere, I can help find the right incantation if you’re
>>> still having trouble.
>>>
>>> In the end my temporary solution was to add this stanza
>>> to org.graalvm.compiler.hotspot.CompilerConfigurationFactory.DefaultBackendMap's
>>> constructor.
>>>
>>> --8<--
>>> String backendFactoryClassName =
>>> System.getProperty("graal.backend.factory"); //
>>> org.grfstuff.compiler.GrfStuffHotSpotBackendFactory
>>> if (backendFactoryClassName != null){
>>> try {
>>> Class<? extends HotSpotBackendFactory> factoryClass
>>> = (Class<? extends HotSpotBackendFactory>)
>>> Class.forName(backendFactoryClassName);
>>> HotSpotBackendFactory factory =
>>> factoryClass.getDeclaredConstructor().newInstance();
>>> Class<? extends Architecture> arch =
>>> factory.getArchitecture();
>>> HotSpotBackendFactory oldEntry =
>>> backends.put(arch, factory);
>>> }catch(Exception e){
>>>
>>> }
>>> }else {
>>> //existing code
>>> }
>>> -->8--
>>>
>>> Then I can build a jdk13 image and run using a combination of
>>> --add-modules along with
>>> -Dgraal.backend.factory=org.grfstuff.compiler.GrfStuffHotSpotBackendFactory
>>>
>>> Whilst this works for me. I am quite sure if our common goal is to get
>>> users to 'adopt/play-with' Graal, we can't expect them to build a dedicated
>>> OpenJDK image.
>>>
>>> I think it is an oversite to offer a ServiceProvider extension mechanism
>>> (HotSpotBackendFactory) only for the module system to forbid anyone from
>>> using it.
>>>
>>>
>>> This is a constraint imposed on us by the fact that Graal is part of the
>>> JDK. Just like Unsafe, JVMCI and Graal cannot be publicly exported. I think
>>> the inconvenience of adding a few --add-exports and --add-reads options to
>>> javac and java is an acceptable overhead to experiment with Graal. We just
>>> need to get the incantation right and hope someone writes a nice blog
>>> article describing it ;-)
>>>
>>> It would be great if we could offer a simpler mechanism (without
>>> resorting to --patch-module which does not seem to be supported by
>>> IntelliJ).
>>>
>>> We already have some protection by forcing
>>> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler maybe we can just
>>> open this up a little more behind these current flags.
>>>
>>>
>>> As stated above, the JVMCI and Graal modules cannot be opened up by
>>> default.
>>>
>>> -Doug
>>>
>>>
>>>
>>> On Mon, Apr 20, 2020 at 1:52 PM Doug Simon <doug.simon at oracle.com>
>>> wrote:
>>>
>>>> Hi Gary,
>>>>
>>>> On 20 Apr 2020, at 14:44, Gary Frost <frost.gary at gmail.com> wrote:
>>>>
>>>> Yeah I started created a module, but it seems for my module to compile
>>>> I need jdk.internal.vm.ci
>>>> <https://urldefense.com/v3/__http://jdk.internal.vm.ci__;!!GqivPVa7Brio!IQVX1iggkDspLlkVBXNQ7IulcpTkED_VhK6SjF2apyR4-XW-E7oZHqeQPPY4GJB0$>
>>>> module to export org.graalvm.compiler.hotspot.HotSpotBackendFactory
>>>> to my named module. Which seems wrong.
>>>>
>>>>
>>>> Actually, that’s by design. Only standard JDK API can be exported
>>>> unqualified from the JDK. Neither Graal nor JVMCI is public API and so
>>>> --add-exports is needed for compilation.
>>>>
>>>>
>>>> I wanted to avoid using GraalVM. I figured I should be able to do what
>>>> I want, using stock/released JDK13 and -XX:+UnlockExperimentalVMOptions
>>>> -XX:+UseJVMCICompiler
>>>>
>>>>
>>>> You can - you just need --add-exports (and maybe some other options) to
>>>> “open up” the classes you’re trying to hook into.
>>>>
>>>> I can 'get around it' by pulling the source for
>>>> org.graalvm.compiler.hotspot.CompilerConfigurationFactory locally. Hacking
>>>> this one file (to load my Factory) then building using
>>>> using --patch-module
>>>> jdk.internal.vm.compiler=src/jdk.internal.vm.compiler/java, but this is
>>>> clearly hacky. I should not have to use --patch-module here.
>>>>
>>>> Alas using --patch-module renders IntelliJ next to useless (it ignores
>>>> patch-module) as pointed out by John Rose and myself here.
>>>> https://youtrack.jetbrains.com/issue/IDEA-224342
>>>> <https://urldefense.com/v3/__https://youtrack.jetbrains.com/issue/IDEA-224342__;!!GqivPVa7Brio!IQVX1iggkDspLlkVBXNQ7IulcpTkED_VhK6SjF2apyR4-XW-E7oZHqeQPKLKl6Nu$>
>>>>
>>>> Maybe the intent was that the JDK versions of Graal, were just to 'use
>>>> Graal', not to allow developers to add there own features.
>>>>
>>>> Maybe that is the distiction between GraalVM and Graal enabled JDK.
>>>>
>>>>
>>>> No, both of these don’t not offer open extension points. You need to
>>>> force it open on the command as mentioned above. We can’t have just any old
>>>> app code getting their hands on the compiler ;-)
>>>>
>>>> -Doug
>>>>
>>>> On Mon, Apr 20, 2020 at 11:54 AM Doug Simon <doug.simon at oracle.com>
>>>> wrote:
>>>>
>>>>> Hi Gary,
>>>>>
>>>>> Have you tried creating a module for your service? I think that’s
>>>>> required as of JDK 9.
>>>>>
>>>>> I suggest opening a PR on https://github.com/oracle/graal
>>>>> <https://urldefense.com/v3/__https://github.com/oracle/graal__;!!GqivPVa7Brio!IQVX1iggkDspLlkVBXNQ7IulcpTkED_VhK6SjF2apyR4-XW-E7oZHqeQPMQSQQsk$>
>>>>> so we can have a more concrete discussion.
>>>>>
>>>>> -Doug
>>>>>
>>>>> > On 20 Apr 2020, at 11:17, Gary Frost <frost.gary at gmail.com> wrote:
>>>>> >
>>>>> > I feel I am missing something obvious ;)
>>>>> >
>>>>> > I am working (well playing actually) with Graal and JDK13 and have
>>>>> built my
>>>>> > own implementation of
>>>>> org.graalvm.compiler.hotspot.HotSpotBackendFactory to
>>>>> > inject some custom code into generated x86. Very cool.
>>>>> >
>>>>> > If I hack the
>>>>> org.graalvm.compiler.hotspot.CompilerConfigurationFactory
>>>>> > class (from JDK13 source) I can instantiate my factory directly,
>>>>> and I see
>>>>> > my modified code.
>>>>> >
>>>>> > But of course I want to able to build against and use use my factory
>>>>> using
>>>>> > a clean opendjdk JDK13 binary distribution.
>>>>> >
>>>>> > It looks like the
>>>>> > class org.graalvm.compiler.hotspot.CompilerConfigurationFactory is
>>>>> setup to
>>>>> > allow me to offer my factory as a 'service provider' which the
>>>>> > CompilerConfigurationFactory accesses via this call
>>>>> > GraalServices.load(HotSpotBackendFactory.class)
>>>>> > but nothing I try works.
>>>>> >
>>>>> > It looks like I need to provide a module-info.java class, but of
>>>>> course I
>>>>> > only want my one service to override the default.
>>>>> >
>>>>> > Any suggestions/help oir pointers to docs would be appreciated.
>>>>> >
>>>>> > Gary
>>>>>
>>>>>
>>>> <graal_service_issue.zip>
>>>
>>>
>>>
More information about the graal-dev
mailing list