BackEnd Service Provider.
Gary Frost
frost.gary at gmail.com
Tue Apr 21 13:07:47 UTC 2020
Great, thanks for looking into it.
Lets see what Alan makes of it.
Gary
On Tue, Apr 21, 2020 at 2:01 PM Doug Simon <doug.simon at oracle.com> wrote:
> Hi Gary,
>
> I cannot understand why --add-reads does not work and have reached out to
> Alan Bateman for more insight.
>
> That said, good to know that at least --patch-module works, even if it
> confuses IntelliJ.
>
> -Doug
>
> On 21 Apr 2020, at 14:37, Gary Frost <frost.gary at gmail.com> wrote:
>
> I also added a makefile which uses patch-module which works.
>
> So
> git clone https://github.com/grfrost/graal_service_issue.git
> <https://urldefense.com/v3/__https://github.com/grfrost/graal_service_issue.git__;!!GqivPVa7Brio!MPgSa5o4fTCXmOWLJrnd3I1rzcvvlfvExeWQRQbKs9vH2Mlu1lJtWon8rUEZndc6$>
> cd graal_service_issue
> # vi makefile* and set path to your JDK13 (JDK11 may work... I know some
> API's have changed between 11 and 13)
> make -f makefile-using-module-service-provider
> make -f makefile-with-patch-module
>
>
> On Tue, Apr 21, 2020 at 12:23 PM Gary Frost <frost.gary at gmail.com> wrote:
>
>> Here is a public github project
>> https://github.com/grfrost/graal_service_issue
>> <https://urldefense.com/v3/__https://github.com/grfrost/graal_service_issue__;!!GqivPVa7Brio!MPgSa5o4fTCXmOWLJrnd3I1rzcvvlfvExeWQRQbKs9vH2Mlu1lJtWon8rXvKqJfe$>
>>
>> git clone https://github.com/grfrost/graal_service_issue.git
>> <https://urldefense.com/v3/__https://github.com/grfrost/graal_service_issue.git__;!!GqivPVa7Brio!MPgSa5o4fTCXmOWLJrnd3I1rzcvvlfvExeWQRQbKs9vH2Mlu1lJtWon8rUEZndc6$>
>> cd graal_service_issue
>> # vi Makefile and set path to your JDK13 (JDK11 may work... I know some
>> API's have changed between 11 and 13)
>> make
>>
>> Gary
>>
>>
>>
>>
>> On Tue, Apr 21, 2020 at 11:59 AM Gary Frost <frost.gary at gmail.com> wrote:
>>
>>> I apologise Doug
>>>
>>> My argument is that there is no combination of '-add-exports and
>>> --add-reads options to javac and java' that will work here. Try it! ;)
>>>
>>> I do/did in fact have it at runtime as well. I stripped it from my
>>> description because it seems to make no difference.
>>>
>>> /usr/lib/jvm/jdk-13/bin/java \
>>> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler
>>> -XX:-TieredCompilation\
>>>
>>> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\
>>>
>>> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\
>>> --add-exports
>>> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\
>>> --add-exports
>>> jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\
>>> --add-exports
>>> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\
>>> -p build -m org.grfstuff.compiler\
>>> -classpath build/org.grfstuff.violajones\
>>> org.grfstuff.violajones.ViolaJones
>>>
>>> I attached a zip of the project to my last email and it got stripped.
>>>
>>> My argument is that there is no combination of '-add-exports and
>>> --add-reads options to javac and java' that will work.
>>>
>>> The only option is --patch-module which is not even officially offered
>>> by javac or java and is not supported by IDE's
>>>
>>> $ /usr/lib/jvm/jdk-13/bin/javac --help | grep patch | wc -l
>>> 0
>>> $ /usr/lib/jvm/jdk-13/bin/java --help | grep patch | wc -l
>>> 0
>>>
>>> I only know of this option because I read the source code ;)
>>>
>>> Gary
>>>
>>> On Tue, Apr 21, 2020 at 11:47 AM Doug Simon <doug.simon at oracle.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On 21 Apr 2020, at 12:30, Gary Frost <frost.gary at gmail.com> wrote:
>>>>
>>>> I don't want to flog this horse excessively, but please humour me one
>>>> more time.
>>>>
>>>> Even after defining a module I can't offer my own service provider for
>>>> org.graalvm.compiler.hotspot.HotSpotBackendFactory
>>>>
>>>> I really think we need a way for folks to experiment with Graal
>>>> backends without having to build there own JDK/JVM.
>>>>
>>>> I created a module to be able to map my service provider
>>>>
>>>> src/org.grfstuff.compiler/
>>>> ├── module-info.java
>>>> └── org
>>>> └── grfstuff
>>>> └── compiler
>>>> └── GrfStuffHotSpotBackendFactory.java
>>>>
>>>>
>>>> Where my module-info.java is this.
>>>> --8<--
>>>> module org.grfstuff.compiler {
>>>> requires jdk.internal.vm.compiler;
>>>> provides org.graalvm.compiler.hotspot.HotSpotBackendFactory with
>>>> org.grfstuff.compiler.GrfStuffHotSpotBackendFactory;
>>>> }
>>>> -->8--
>>>>
>>>> My implementation is just enough code so I can see an instance of my
>>>> backend get loaded.
>>>> --8<--
>>>> package org.grfstuff.compiler;
>>>> import org.graalvm.compiler.hotspot.HotSpotBackendFactory;
>>>> import org.graalvm.compiler.serviceprovider.ServiceProvider;
>>>> import org.graalvm.compiler.hotspot.amd64.AMD64HotSpotBackendFactory;
>>>>
>>>> @ServiceProvider(HotSpotBackendFactory.class)
>>>> public class GrfStuffHotSpotBackendFactory extends
>>>> AMD64HotSpotBackendFactory {
>>>> public GrfStuffHotSpotBackendFactory(){
>>>> System.out.println("GrfStuffHotSpotFactory");
>>>> }
>>>> public String toString() {
>>>> return "GRFSTUFF_AMD64";
>>>> }
>>>> }
>>>> -->8--
>>>>
>>>> I can build my module using appropriate --add-exports 'jiggery pokery '
>>>> :)
>>>>
>>>> javac -g \
>>>> -d build \
>>>> --add-exports
>>>> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\
>>>> --add-exports
>>>> jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\
>>>> --add-exports
>>>> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\
>>>> --module-source-path src\
>>>> --module org.grfstuff.compiler\
>>>> $(shell find src/org.grfstuff.compiler -name *.java)
>>>>
>>>> But at runtime (my app is a compute intensive voilajones face detection
>>>> algorithm)
>>>> java \
>>>> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler
>>>> -XX:-TieredCompilation\
>>>>
>>>> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\
>>>>
>>>> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\
>>>> -p build -m org.grfstuff.compiler\
>>>> -classpath build/org.grfstuff.violajones\
>>>> org.grfstuff.violajones.ViolaJones
>>>>
>>>> I get this error
>>>>
>>>> Error occurred during initialization of boot layer
>>>> java.lang.module.ResolutionException: Module org.grfstuff.compiler does
>>>> not read a module that exports org.graalvm.compiler.hotspot
>>>>
>>>> It would appear that whilst add-exports is enough at compile time to
>>>> persuade the compiler that all is well. At runtime the service loader
>>>> execution path does not honour the add-exports requests to the service
>>>> layer.
>>>>
>>>>
>>>> If you use —add-exports during java compilation, you almost certainly
>>>> need to use it at runtime. That is, repeat your --add-export arguments to
>>>> the java launcher. You may also need some --add-reads options. If you put
>>>> you jars up somewhere, I can help find the right incantation if you’re
>>>> still having trouble.
>>>>
>>>> In the end my temporary solution was to add this stanza
>>>> to org.graalvm.compiler.hotspot.CompilerConfigurationFactory.DefaultBackendMap's
>>>> constructor.
>>>>
>>>> --8<--
>>>> String backendFactoryClassName =
>>>> System.getProperty("graal.backend.factory"); //
>>>> org.grfstuff.compiler.GrfStuffHotSpotBackendFactory
>>>> if (backendFactoryClassName != null){
>>>> try {
>>>> Class<? extends HotSpotBackendFactory>
>>>> factoryClass = (Class<? extends HotSpotBackendFactory>)
>>>> Class.forName(backendFactoryClassName);
>>>> HotSpotBackendFactory factory =
>>>> factoryClass.getDeclaredConstructor().newInstance();
>>>> Class<? extends Architecture> arch =
>>>> factory.getArchitecture();
>>>> HotSpotBackendFactory oldEntry =
>>>> backends.put(arch, factory);
>>>> }catch(Exception e){
>>>>
>>>> }
>>>> }else {
>>>> //existing code
>>>> }
>>>> -->8--
>>>>
>>>> Then I can build a jdk13 image and run using a combination of
>>>> --add-modules along with
>>>> -Dgraal.backend.factory=org.grfstuff.compiler.GrfStuffHotSpotBackendFactory
>>>>
>>>> Whilst this works for me. I am quite sure if our common goal is to get
>>>> users to 'adopt/play-with' Graal, we can't expect them to build a dedicated
>>>> OpenJDK image.
>>>>
>>>> I think it is an oversite to offer a ServiceProvider extension
>>>> mechanism (HotSpotBackendFactory) only for the module system to forbid
>>>> anyone from using it.
>>>>
>>>>
>>>> This is a constraint imposed on us by the fact that Graal is part of
>>>> the JDK. Just like Unsafe, JVMCI and Graal cannot be publicly exported. I
>>>> think the inconvenience of adding a few --add-exports and --add-reads
>>>> options to javac and java is an acceptable overhead to experiment with
>>>> Graal. We just need to get the incantation right and hope someone writes a
>>>> nice blog article describing it ;-)
>>>>
>>>> It would be great if we could offer a simpler mechanism (without
>>>> resorting to --patch-module which does not seem to be supported by
>>>> IntelliJ).
>>>>
>>>> We already have some protection by forcing
>>>> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler maybe we can just
>>>> open this up a little more behind these current flags.
>>>>
>>>>
>>>> As stated above, the JVMCI and Graal modules cannot be opened up by
>>>> default.
>>>>
>>>> -Doug
>>>>
>>>>
>>>>
>>>> On Mon, Apr 20, 2020 at 1:52 PM Doug Simon <doug.simon at oracle.com>
>>>> wrote:
>>>>
>>>>> Hi Gary,
>>>>>
>>>>> On 20 Apr 2020, at 14:44, Gary Frost <frost.gary at gmail.com> wrote:
>>>>>
>>>>> Yeah I started created a module, but it seems for my module to compile
>>>>> I need jdk.internal.vm.ci
>>>>> <https://urldefense.com/v3/__http://jdk.internal.vm.ci__;!!GqivPVa7Brio!IQVX1iggkDspLlkVBXNQ7IulcpTkED_VhK6SjF2apyR4-XW-E7oZHqeQPPY4GJB0$>
>>>>> module to export org.graalvm.compiler.hotspot.HotSpotBackendFactory
>>>>> to my named module. Which seems wrong.
>>>>>
>>>>>
>>>>> Actually, that’s by design. Only standard JDK API can be exported
>>>>> unqualified from the JDK. Neither Graal nor JVMCI is public API and so
>>>>> --add-exports is needed for compilation.
>>>>>
>>>>>
>>>>> I wanted to avoid using GraalVM. I figured I should be able to do
>>>>> what I want, using stock/released JDK13 and
>>>>> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler
>>>>>
>>>>>
>>>>> You can - you just need --add-exports (and maybe some other options)
>>>>> to “open up” the classes you’re trying to hook into.
>>>>>
>>>>> I can 'get around it' by pulling the source for
>>>>> org.graalvm.compiler.hotspot.CompilerConfigurationFactory locally. Hacking
>>>>> this one file (to load my Factory) then building using
>>>>> using --patch-module
>>>>> jdk.internal.vm.compiler=src/jdk.internal.vm.compiler/java, but this is
>>>>> clearly hacky. I should not have to use --patch-module here.
>>>>>
>>>>> Alas using --patch-module renders IntelliJ next to useless (it ignores
>>>>> patch-module) as pointed out by John Rose and myself here.
>>>>> https://youtrack.jetbrains.com/issue/IDEA-224342
>>>>> <https://urldefense.com/v3/__https://youtrack.jetbrains.com/issue/IDEA-224342__;!!GqivPVa7Brio!IQVX1iggkDspLlkVBXNQ7IulcpTkED_VhK6SjF2apyR4-XW-E7oZHqeQPKLKl6Nu$>
>>>>>
>>>>> Maybe the intent was that the JDK versions of Graal, were just to 'use
>>>>> Graal', not to allow developers to add there own features.
>>>>>
>>>>> Maybe that is the distiction between GraalVM and Graal enabled JDK.
>>>>>
>>>>>
>>>>> No, both of these don’t not offer open extension points. You need to
>>>>> force it open on the command as mentioned above. We can’t have just any old
>>>>> app code getting their hands on the compiler ;-)
>>>>>
>>>>> -Doug
>>>>>
>>>>> On Mon, Apr 20, 2020 at 11:54 AM Doug Simon <doug.simon at oracle.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Gary,
>>>>>>
>>>>>> Have you tried creating a module for your service? I think that’s
>>>>>> required as of JDK 9.
>>>>>>
>>>>>> I suggest opening a PR on https://github.com/oracle/graal
>>>>>> <https://urldefense.com/v3/__https://github.com/oracle/graal__;!!GqivPVa7Brio!IQVX1iggkDspLlkVBXNQ7IulcpTkED_VhK6SjF2apyR4-XW-E7oZHqeQPMQSQQsk$>
>>>>>> so we can have a more concrete discussion.
>>>>>>
>>>>>> -Doug
>>>>>>
>>>>>> > On 20 Apr 2020, at 11:17, Gary Frost <frost.gary at gmail.com> wrote:
>>>>>> >
>>>>>> > I feel I am missing something obvious ;)
>>>>>> >
>>>>>> > I am working (well playing actually) with Graal and JDK13 and have
>>>>>> built my
>>>>>> > own implementation of
>>>>>> org.graalvm.compiler.hotspot.HotSpotBackendFactory to
>>>>>> > inject some custom code into generated x86. Very cool.
>>>>>> >
>>>>>> > If I hack the
>>>>>> org.graalvm.compiler.hotspot.CompilerConfigurationFactory
>>>>>> > class (from JDK13 source) I can instantiate my factory directly,
>>>>>> and I see
>>>>>> > my modified code.
>>>>>> >
>>>>>> > But of course I want to able to build against and use use my
>>>>>> factory using
>>>>>> > a clean opendjdk JDK13 binary distribution.
>>>>>> >
>>>>>> > It looks like the
>>>>>> > class org.graalvm.compiler.hotspot.CompilerConfigurationFactory is
>>>>>> setup to
>>>>>> > allow me to offer my factory as a 'service provider' which the
>>>>>> > CompilerConfigurationFactory accesses via this call
>>>>>> > GraalServices.load(HotSpotBackendFactory.class)
>>>>>> > but nothing I try works.
>>>>>> >
>>>>>> > It looks like I need to provide a module-info.java class, but of
>>>>>> course I
>>>>>> > only want my one service to override the default.
>>>>>> >
>>>>>> > Any suggestions/help oir pointers to docs would be appreciated.
>>>>>> >
>>>>>> > Gary
>>>>>>
>>>>>>
>>>>> <graal_service_issue.zip>
>>>>
>>>>
>>>>
>
More information about the graal-dev
mailing list