JDK11 & Polyglot
David Lloyd
david.lloyd at redhat.com
Thu May 23 19:17:57 UTC 2019
I think that if/when JPMS support comes in, it'll probably substitute
all these classes away with native image focused versions, similarly
perhaps to how classes are done. So IMO this kind of fix is just
fine.
On Thu, May 23, 2019 at 12:41 PM Bob McWhirter <bmcwhirt at redhat.com> wrote:
>
> And my "solution":
>
> https://github.com/oracle/graal/pull/1326
>
> On Thu, May 23, 2019 at 11:55 AM Bob McWhirter <bmcwhirt at redhat.com> wrote:
>
> > Progress... maybe?
> >
> > In Module$ReflectionData
> >
> > /**
> > * A module (1st key) exports or opens a package to another module
> > * (2nd key). The map value is a map of package name to a boolean
> > * that indicates if the package is opened.
> > */
> > static final WeakPairMap<Module, Module, Map<String, Boolean>> exports =
> > new WeakPairMap<>();
> >
> >
> > Ultimately used by Module.isReflectivelyExportedOrOpen(...):
> >
> > private boolean isReflectivelyExportedOrOpen(String pn, Module other, boolean open) {
> > // exported or open to all modules
> > Map<String, Boolean> exports = ReflectionData.exports.get(this, EVERYONE_MODULE);
> >
> >
> >
> > Is the source of things-changing-while-writing-the-heap.
> >
> > Not sure how to fix, or if there is a way to whitelist/ignore this.
> >
> > I do have an inbound PR to push in a moment helping fix the debug reason
> > trace, to avoid the "root=null" in the current master#HEAD.
> >
> > Caused by: com.oracle.svm.core.util.UserError$UserException: Static field
> > or an object referenced from a static field changed during native image
> > generation?
> > object:com.oracle.truffle.api=true of class:
> > java.util.concurrent.ConcurrentHashMap$Node
> > reachable through:
> > object: [Ljava.util.concurrent.ConcurrentHashMap$Node;@3cb34219 of
> > class: java.util.concurrent.ConcurrentHashMap$Node[]
> > object: {com.oracle.truffle.api.dsl=true,
> > com.oracle.truffle.api.library=true, com.oracle.truffle.api.nodes=true,
> > com.oracle.truffle.api=true} of class:
> > java.util.concurrent.ConcurrentHashMap
> > object: java.lang.WeakPairMap$Pair$Weak at 248dd9c1={com.oracle.truffle.api.dsl=true,
> > com.oracle.truffle.api.library=true, com.oracle.truffle.api.nodes=true,
> > com.oracle.truffle.api=true} of class:
> > java.util.concurrent.ConcurrentHashMap$Node
> > object: java.lang.WeakPairMap$Pair$Weak at 40620d6e={jdk.internal.vm.annotation=false,
> > jdk.internal=false, jdk.internal.reflect=false} of class:
> > java.util.concurrent.ConcurrentHashMap$Node
> > object: java.lang.WeakPairMap$Pair$Weak at 40620d6e={com.sun.proxy.jdk.proxy1=false}
> > of class: java.util.concurrent.ConcurrentHashMap$Node
> > object: [Ljava.util.concurrent.ConcurrentHashMap$Node;@6e0bf27b of
> > class: java.util.concurrent.ConcurrentHashMap$Node[]
> > object: {java.lang.WeakPairMap$Pair$Weak at 64486d2a={org.graalvm.nativeimage.c.constant=true},
> > java.lang.WeakPairMap$Pair$Weak at 708aa969={jdk.vm.ci.aarch64=true,
> > jdk.vm.ci.services=true, jdk.vm.ci.common=true, jdk.vm.ci.code=true,
> > jdk.vm.ci.hotspot.aarch64=true, jdk.vm.ci.runtime=true,
> > jdk.vm.ci.hotspot.sparc=true, jdk.vm.ci.code.stack=true,
> > jdk.vm.ci.hotspot.amd64=true, jdk.vm.ci.meta=true,
> > jdk.vm.ci.code.site=true, jdk.vm.ci.sparc=true, jdk.vm.ci.hotspot=true,
> > jdk.vm.ci.services.internal=true, jdk.vm.ci.amd64=true},
> > java.lang.WeakPairMap$Pair$Weak at 6f715c77={jdk.vm.ci.common=false},
> > java.lang.WeakPairMap$Pair$Weak at 40620d6e={com.sun.proxy.jdk.proxy1=false},
> > java.lang.WeakPairMap$Pair$Weak at 40620d6e={jdk.internal.vm.annotation=false,
> > jdk.internal=false, jdk.internal.reflect=false},
> > java.lang.WeakPairMap$Pair$Weak at 248dd9c1={com.oracle.truffle.api.dsl=true,
> > com.oracle.truffle.api.library=true, com.oracle.truffle.api.nodes=true,
> > com.oracle.truffle.api=true}, java.lang.WeakPairMap$Pair$Weak at 7c968d5b={org.graalvm.compiler.debug=true,
> > org.graalvm.compiler.nodes=true}, java.lang.WeakPairMap$Pair$Weak at 5ba3fc8d={jdk.vm.ci.aarch64=true,
> > jdk.vm.ci.services=true, jdk.vm.ci.common=true, jdk.vm.ci.code=true,
> > jdk.vm.ci.hotspot.aarch64=true, jdk.vm.ci.runtime=true,
> > jdk.vm.ci.hotspot.sparc=true, jdk.vm.ci.code.stack=true,
> > jdk.vm.ci.hotspot.amd64=true, jdk.vm.ci.meta=true,
> > jdk.vm.ci.code.site=true, jdk.vm.ci.sparc=true, jdk.vm.ci.hotspot=true,
> > jdk.vm.ci.services.internal=true, jdk.vm.ci.amd64=true},
> > java.lang.WeakPairMap$Pair$Weak at 6112b142={com.oracle.truffle.api.impl=true,
> > com.oracle.truffle.polyglot=true}, java.lang.WeakPairMap$Pair$Weak at 3909e5d8={org.graalvm.compiler.graph=true,
> > org.graalvm.compiler.core.match=true},
> > java.lang.WeakPairMap$Pair$Weak at 49834cb1={jdk.vm.ci.aarch64=true,
> > jdk.vm.ci.services=true, jdk.vm.ci.common=true, jdk.vm.ci.code=true,
> > jdk.vm.ci.hotspot.aarch64=true, jdk.vm.ci.runtime=true,
> > jdk.vm.ci.hotspot.sparc=true, jdk.vm.ci.code.stack=true,
> > jdk.vm.ci.hotspot.amd64=true, jdk.vm.ci.meta=true,
> > jdk.vm.ci.code.site=true, jdk.vm.ci.sparc=true, jdk.vm.ci.hotspot=true,
> > jdk.vm.ci.services.internal=true, jdk.vm.ci.amd64=true},
> > java.lang.WeakPairMap$Pair$Weak at 2cc8e91b={org.graalvm.word.impl=false},
> > java.lang.WeakPairMap$Pair$Weak at 1a0690f3={java.util=true, java.nio=true,
> > jdk.internal.misc=true, jdk.internal.logger=true, java.lang=true,
> > java.lang.ref=true, jdk.internal.vm.annotation=true, java.net=true,
> > java.lang.invoke=true, jdk.internal.ref=true, jdk.internal.module=true,
> > jdk.internal.reflect=true}, java.lang.WeakPairMap$Pair$Weak at 17197c41={com.sun.proxy.jdk.proxy2=false},
> > java.lang.WeakPairMap$Pair$Weak at 21d705a9={org.graalvm.nativeimage.impl=true,
> > org.graalvm.polyglot=true}, java.lang.WeakPairMap$Pair$Weak at 3515c1ea={jdk.vm.ci.aarch64=false,
> > jdk.vm.ci.meta=false, jdk.vm.ci.code.site=false, jdk.vm.ci.services=false,
> > jdk.vm.ci.code=false, jdk.vm.ci.common=false, jdk.vm.ci.hotspot=false,
> > jdk.vm.ci.runtime=false, jdk.vm.ci.amd64=false}} of class:
> > java.util.concurrent.ConcurrentHashMap
> > object: java.lang.WeakPairMap at 3b3c322f of class:
> > java.lang.WeakPairMap
> > root: java.lang.Module.isReflectivelyExportedOrOpen(String, Module,
> > boolean)
> >
> >
> >
> > On Thu, May 23, 2019 at 9:19 AM Bob McWhirter <bmcwhirt at redhat.com> wrote:
> >
> >> Seems like an intermittent/transient failure, as this morning it all
> >> built cleanly.
> >>
> >> Good times.
> >>
> >> -Bob
> >>
> >> On Thu, May 23, 2019 at 6:36 AM Bob McWhirter <bmcwhirt at redhat.com>
> >> wrote:
> >>
> >>> Thanks Andrew!
> >>>
> >>> I’ll continue to poke around. When I was getting the ObjectIdentifier
> >>> variant there simple aren’t many statics available. There’s a none-static
> >>> stringForm field of type String. Maybe I’m just misunderstanding the error
> >>> message also.
> >>>
> >>> Time to git log.
> >>>
> >>> Bob
> >>>
> >>> On Thu, May 23, 2019 at 5:02 AM Andrew Dinn <adinn at redhat.com> wrote:
> >>>
> >>>> On 22/05/2019 19:20, Bob McWhirter wrote:
> >>>> > With the new JDK11 changes, at one point I could build
> >>>> polyglot/libpolyglot
> >>>> > with JDK11.
> >>>> >
> >>>> > Currently, though, I'm hitting:
> >>>> > . . .
> >>>> > I sense that it might be the transient static boolean COMPACT_STRINGS
> >>>> on
> >>>> > String.class ultimately causing it, but that's not necessarily a
> >>>> warranted
> >>>> > opinion. It always seems to be a String problem, and COMPACT_STRINGS
> >>>> is
> >>>> > designated as set by the JVM at runtime, not to the true that appears
> >>>> in
> >>>> > the source.
> >>>>
> >>>> I'm not sure how this could be the issue. I say that because I do
> >>>> understand what goes on with this field in the JVM. Here is the context:
> >>>>
> >>>> During JVM startup various Java classes are constructed 'by hand' i.e.
> >>>>
> >>>> the bytecodes are loaded
> >>>> the JVM-internal metadata model for the class is created
> >>>> (C++ object InstanceKlass plus auxiliary objects like Symbol. method
> >>>> etc)
> >>>> the JVM-heap allocated Class<?> is created
> >>>> the Class<?>'s <clinit> method is run
> >>>>
> >>>> This is done in a very specific sequence starting with some Exception
> >>>> classes that have no dependencies on supers or linked classes and
> >>>> proceeding to classes like String, Thread, ThreadGroup etc.
> >>>>
> >>>> At certain points during this process a <clinit> run may require
> >>>> creating certain linked Class<?> instances (most obviously supers but
> >>>> also classes referenced from the <clinit> method) and this will initiate
> >>>> the above steps recursively. So, the explicit hand initialization
> >>>> sequence only defines a nominal load+init order. Recursive dependency
> >>>> chasing means that the actual load order may be different.
> >>>>
> >>>> Now as for class String: very early on in this init sequence String gets
> >>>> explicitly loaded. It is one of many classes whose static fields need to
> >>>> be initialized specially to accord with HW, OS or user config setting --
> >>>> String.COMPACT_STRINGS is one such example. In most normal cases where
> >>>> runtime-specific settings need to be detected the <clinit> code calls
> >>>> native methods from <clinit> code to read them. However, for a few of
> >>>> these early-load classes the <clinit> methods instead initialize to a
> >>>> dummy and the C++ code that hand-loads the class then overwrites the
> >>>> correct values into the static field slots immediately after the load
> >>>> and init of the class has completed.
> >>>>
> >>>> So, in the case of String field COMPACT_STRINGS will be updated by the
> >>>> JVM immediately after running its <clinit> method. I don't really see
> >>>> how this can make any difference visible to the Graal native image
> >>>> generator. The change to COMPACT_STRINGS wil have happened long before
> >>>> any of the Graal NativeImageGenerator code could even have started
> >>>> running (indeed well before the JVM runtime code like ForkJoinPool that
> >>>> drives the native image process is loaded). So, sorry Bob, but I think
> >>>> you are probably barking up the wrong tree here.
> >>>>
> >>>> > Anyone have a thought or a pointer on what might've changed in the
> >>>> past 3
> >>>> > weeks, or if I'm completely off base and would be willing to point me
> >>>> in a
> >>>> > more useful direction.
> >>>> Looking at recent changes sounds to me like the best approach.
> >>>>
> >>>> regards,
> >>>>
> >>>>
> >>>> Andrew Dinn
> >>>> -----------
> >>>> Senior Principal Software Engineer
> >>>> Red Hat UK Ltd
> >>>> Registered in England and Wales under Company Registration No. 03798903
> >>>> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander
> >>>>
> >>>
--
- DML
More information about the graal-dev
mailing list