RFR (L): 8003868: fix shark for latest HotSpot and LLVM [Was: Re: RFR: Fix shark for latest Hotspot and LLVM]
Roman Kennke
rkennke at redhat.com
Fri Nov 23 09:14:48 PST 2012
Am Mittwoch, den 21.11.2012, 14:52 -0800 schrieb Christian Thalinger:
> On Nov 21, 2012, at 2:22 PM, Christian Thalinger <christian.thalinger at oracle.com> wrote:
>
> >
> > On Nov 21, 2012, at 2:17 PM, Christian Thalinger <christian.thalinger at oracle.com> wrote:
> >
> >>
> >> On Nov 21, 2012, at 12:54 PM, Roman Kennke <rkennke at redhat.com> wrote:
> >>
> >>> Am Mittwoch, den 21.11.2012, 12:47 -0800 schrieb Christian Thalinger:
> >>>> On Nov 21, 2012, at 11:43 AM, Christian Thalinger <christian.thalinger at oracle.com> wrote:
> >>>>
> >>>>> On Nov 21, 2012, at 9:31 AM, Roman Kennke <rkennke at redhat.com> wrote:
> >>>>>
> >>>>>> Hi there,
> >>>>>>
> >>>>>> during the last days I worked on fixing the Shark compiler for Hotspot
> >>>>>> to get it to build and run again, with the latest Hotspot code and LLVM.
> >>>>>> Here are some details:
> >>>>>>
> >>>>>> - A lot of changes are just to make it build and the compiler happy. For
> >>>>>> example, I had to remove a lot of 'const' qualifiers because of API
> >>>>>> changes in LLVM.
> >>>>>> - Most other changes have to do with the split of the oop and metadata
> >>>>>> class hierarchies in Hotspot.
> >>>>>> - Then there have been a few changes caused by LLVM changes and
> >>>>>> improvements, most notably the LLVM intrinsics for atomic operations
> >>>>>> (memory barrier and cmpxchg) have been removed and now have a
> >>>>>> representation directly in LLVM's IR. This makes our code a little
> >>>>>> nicer.
> >>>>>>
> >>>>>> I tested this by running a number of applications, most notably Eclipse
> >>>>>> (which is notoriously difficult on VMs), Java2Demo, SwingSet2 and a
> >>>>>> bunch of other stuff.
> >>>>>>
> >>>>>> I would like to get this integrated into OpenJDK now if possible. You
> >>>>>> can find the full webrev here:
> >>>>>>
> >>>>>> http://cr.openjdk.java.net/~rkennke/shark/webrev.00/
> >>>>>
> >>>>> The changes seem to touch almost only shark files so these should be fine. One question though:
> >>>>>
> >>>>> + develop(bool, SharkShowCompiledMethods, false, \
> >>>>>
> >>>>> Isn't PrintCompilation doing that already?
> >>>>>
> >>>>> The shared code changes look good. I filed:
> >>>>>
> >>>>> 8003868: fix shark for latest HotSpot and LLVM
> >>>>>
> >>>>> -- Chris
> >>>>>
> >>>>>>
> >>>>>> There are also a very minor change required in JDK:
> >>>>>>
> >>>>>> http://cr.openjdk.java.net/~rkennke/shark/webrev-jdk-00/
> >>>>>>
> >>>>>> In order to build it, apply the patches on hsx/hotspot-comp 's hotspot
> >>>>>> and jdk repositories respectivly. Find my build script here:
> >>>>>>
> >>>>>> http://cr.openjdk.java.net/~rkennke/shark/Build8-zero-shark
> >>>>>>
> >>>>>> (Review and adjust variables to your settings, most notably you will
> >>>>>> need to change LLVM_CONFIG to point to your LLVM 3.1 installation.)
> >>>>>>
> >>>>>> Please let me know if there are any issues or how we can get this
> >>>>>> integrated into Hotspot.
> >>>>
> >>>> Finally I installed LLVM on one of our machines to be able to do a Shark build once in a while. When I try to do a jvmgshark build I get:
> >>>>
> >>>> In file included from /usr/local/include/llvm/Support/PointerLikeTypeTraits.h:18,
> >>>> from /usr/local/include/llvm/ADT/PointerIntPair.h:17,
> >>>> from /usr/local/include/llvm/Use.h:28,
> >>>> from /usr/local/include/llvm/Value.h:17,
> >>>> from /usr/local/include/llvm/Argument.h:17,
> >>>> from /home/cthaling/8003868/src/share/vm/shark/llvmHeaders.hpp:39,
> >>>> from /home/cthaling/8003868/src/share/vm/shark/sharkEntry.hpp:29,
> >>>> from /home/cthaling/8003868/src/share/vm/compiler/disassembler.cpp:51:
> >>>> /usr/local/include/llvm/Support/DataTypes.h:53:3: error: #error "Must #define __STDC_CONSTANT_MACROS before " "#including Support/DataTypes.h"
> >>>>
> >>>> and:
> >>>>
> >>>> In file included from /usr/local/include/llvm/Attributes.h:18,
> >>>> from /usr/local/include/llvm/Argument.h:18,
> >>>> from /home/cthaling/8003868/src/share/vm/shark/llvmHeaders.hpp:39,
> >>>> from /home/cthaling/8003868/src/share/vm/shark/sharkEntry.hpp:29,
> >>>> from /home/cthaling/8003868/src/share/vm/compiler/disassembler.cpp:51:
> >>>> /usr/local/include/llvm/Support/MathExtras.h: In function ‘bool llvm::isInt(int64_t)’:
> >>>> /usr/local/include/llvm/Support/MathExtras.h:38: error: there are no arguments to ‘INT64_C’ that depend on a template parameter, so a declaration of ‘INT64_C’ must be available
> >>>> /usr/local/include/llvm/Support/MathExtras.h:38: note: (if you use ‘-fpermissive’, G++ will accept your code, but allowing the use of an undeclared name is deprecated)
> >>>> /usr/local/include/llvm/Support/MathExtras.h:38: error: there are no arguments to ‘INT64_C’ that depend on a template parameter, so a declaration of ‘INT64_C’ must be available
> >>>> /usr/local/include/llvm/Support/MathExtras.h: In function ‘bool llvm::isUInt(uint64_t)’:
> >>>> /usr/local/include/llvm/Support/MathExtras.h:64: error: there are no arguments to ‘UINT64_C’ that depend on a template parameter, so a declaration of ‘UINT64_C’ must be available
> >>>> /usr/local/include/llvm/Support/MathExtras.h: In function ‘bool llvm::isIntN(unsigned int, int64_t)’:
> >>>> /usr/local/include/llvm/Support/MathExtras.h:96: error: ‘INT64_C’ was not declared in this scope
> >>>>
> >>>> Not sure if the latter is because of the former one. Have you seen this before?
> >>>
> >>> Yes, it's caused by the former. And yes, I have seen it before. IIRC,
> >>> this happens when certain cflags are not set correctly. The script
> >>> jdk/make/jdk_generic_profile.sh will call llvm-config to figure out the
> >>> correct flags. In order for this to work, you need to have the full path
> >>> to the llvm-config script set in the LLVM_CONFIG env variable. Were you
> >>> using the build script that I provided?
> >>
> >> No. I took your script and got the important environment variables. But I missed the LLVM_* ones. Usually we only build hotspot so we don't have this jdk script.
> >>
> >> Now that I have the LLVM_* variables it's even worse:
> >>
> >> /home/cthaling/8003868/src/share/vm/oops/oop.hpp:72: error: cast from type ‘markOopDesc* const volatile*’ to type ‘markOopDesc**’ casts away constness
> >>
> >> It's probably this guy: -Wcast-qual
> >
> > Got it:
> >
> > $ java -version
> > java version "1.8.0-ea"
> > Java(TM) SE Runtime Environment (build 1.8.0-ea-b65)
> > OpenJDK 64-Bit Shark VM (build 25.0-b11-internal-jvmg, mixed mode)
>
> I ran the compiler regression tests and Shark crashes in 5091921:
>
> cthaling at intelsdv03.us.oracle.com:~/8003868/test$ jtreg -workDir:$EXPORTHOME/jtreg -reportDir:$EXPORTHOME/jtreg -testjdk:$JAVA_HOME -verbose:summary compiler/5091921/
> Directory "/export/twisti/jtreg/scratch" not found: creating
> Passed: compiler/5091921/Test5091921.java
> Passed: compiler/5091921/Test6186134.java
> Passed: compiler/5091921/Test6196102.java
> Passed: compiler/5091921/Test6357214.java
> Passed: compiler/5091921/Test6559156.java
> Passed: compiler/5091921/Test6753639.java
> Passed: compiler/5091921/Test6850611.java
> Passed: compiler/5091921/Test6890943.java
> Passed: compiler/5091921/Test6897150.java
> Passed: compiler/5091921/Test6905845.java
> Passed: compiler/5091921/Test6931567.java
> /net/sqenfs-1.us.oracle.com/export1/comp/vm/tool/jtreg/execution/linux/bin/jtreg: line 139: 27784 Segmentation fault (core dumped) "${JT_JAVA}" $javaOpts -Dprogram=`basename "$0"` -jar "${JT_HOME}/lib/jtreg.jar" $jtregOpts
>
> You can also run all them with a simple make in test/ by setting:
>
> PRODUCT_HOME=$JAVA_HOME
> TESTDIRS=compiler
Alright, I found another fairly grave bug that I would like to include a
fix for: apparently, a 'fence' is not enough of a memory barrier for
volatile putfield/getfield (duh). Which basically broke all of j.u.c.
LLVM offers atomic loads and stores, which seem to be exactly what is
needed for volatile field access. However, it does not provide helper
functions for those in llvm::IRBuilder so I wrote my own. This should be
the final patch for now (unless you find something else).
http://cr.openjdk.java.net/~rkennke/shark/webrev.03/
Cheers,
Roman
More information about the hotspot-dev
mailing list