RFR (L): 8003868: fix shark for latest HotSpot and LLVM [Was: Re: RFR: Fix shark for latest Hotspot and LLVM]
Roman Kennke
rkennke at redhat.com
Thu Nov 22 14:00:53 PST 2012
Am Donnerstag, den 22.11.2012, 18:47 +0100 schrieb Roman Kennke:
> Am Donnerstag, den 22.11.2012, 17:48 +0100 schrieb Roman Kennke:
> > Am Mittwoch, den 21.11.2012, 14:52 -0800 schrieb Christian Thalinger:
> > > On Nov 21, 2012, at 2:22 PM, Christian Thalinger <christian.thalinger at oracle.com> wrote:
> > >
> > > >
> > > > On Nov 21, 2012, at 2:17 PM, Christian Thalinger <christian.thalinger at oracle.com> wrote:
> > > >
> > > >>
> > > >> On Nov 21, 2012, at 12:54 PM, Roman Kennke <rkennke at redhat.com> wrote:
> > > >>
> > > >>> Am Mittwoch, den 21.11.2012, 12:47 -0800 schrieb Christian Thalinger:
> > > >>>> On Nov 21, 2012, at 11:43 AM, Christian Thalinger <christian.thalinger at oracle.com> wrote:
> > > >>>>
> > > >>>>> On Nov 21, 2012, at 9:31 AM, Roman Kennke <rkennke at redhat.com> wrote:
> > > >>>>>
> > > >>>>>> Hi there,
> > > >>>>>>
> > > >>>>>> during the last days I worked on fixing the Shark compiler for Hotspot
> > > >>>>>> to get it to build and run again, with the latest Hotspot code and LLVM.
> > > >>>>>> Here are some details:
> > > >>>>>>
> > > >>>>>> - A lot of changes are just to make it build and the compiler happy. For
> > > >>>>>> example, I had to remove a lot of 'const' qualifiers because of API
> > > >>>>>> changes in LLVM.
> > > >>>>>> - Most other changes have to do with the split of the oop and metadata
> > > >>>>>> class hierarchies in Hotspot.
> > > >>>>>> - Then there have been a few changes caused by LLVM changes and
> > > >>>>>> improvements, most notably the LLVM intrinsics for atomic operations
> > > >>>>>> (memory barrier and cmpxchg) have been removed and now have a
> > > >>>>>> representation directly in LLVM's IR. This makes our code a little
> > > >>>>>> nicer.
> > > >>>>>>
> > > >>>>>> I tested this by running a number of applications, most notably Eclipse
> > > >>>>>> (which is notoriously difficult on VMs), Java2Demo, SwingSet2 and a
> > > >>>>>> bunch of other stuff.
> > > >>>>>>
> > > >>>>>> I would like to get this integrated into OpenJDK now if possible. You
> > > >>>>>> can find the full webrev here:
> > > >>>>>>
> > > >>>>>> http://cr.openjdk.java.net/~rkennke/shark/webrev.00/
> > > >>>>>
> > > >>>>> The changes seem to touch almost only shark files so these should be fine. One question though:
> > > >>>>>
> > > >>>>> + develop(bool, SharkShowCompiledMethods, false, \
> > > >>>>>
> > > >>>>> Isn't PrintCompilation doing that already?
> > > >>>>>
> > > >>>>> The shared code changes look good. I filed:
> > > >>>>>
> > > >>>>> 8003868: fix shark for latest HotSpot and LLVM
> > > >>>>>
> > > >>>>> -- Chris
> > > >>>>>
> > > >>>>>>
> > > >>>>>> There are also a very minor change required in JDK:
> > > >>>>>>
> > > >>>>>> http://cr.openjdk.java.net/~rkennke/shark/webrev-jdk-00/
> > > >>>>>>
> > > >>>>>> In order to build it, apply the patches on hsx/hotspot-comp 's hotspot
> > > >>>>>> and jdk repositories respectivly. Find my build script here:
> > > >>>>>>
> > > >>>>>> http://cr.openjdk.java.net/~rkennke/shark/Build8-zero-shark
> > > >>>>>>
> > > >>>>>> (Review and adjust variables to your settings, most notably you will
> > > >>>>>> need to change LLVM_CONFIG to point to your LLVM 3.1 installation.)
> > > >>>>>>
> > > >>>>>> Please let me know if there are any issues or how we can get this
> > > >>>>>> integrated into Hotspot.
> > > >>>>
> > > >>>> Finally I installed LLVM on one of our machines to be able to do a Shark build once in a while. When I try to do a jvmgshark build I get:
> > > >>>>
> > > >>>> In file included from /usr/local/include/llvm/Support/PointerLikeTypeTraits.h:18,
> > > >>>> from /usr/local/include/llvm/ADT/PointerIntPair.h:17,
> > > >>>> from /usr/local/include/llvm/Use.h:28,
> > > >>>> from /usr/local/include/llvm/Value.h:17,
> > > >>>> from /usr/local/include/llvm/Argument.h:17,
> > > >>>> from /home/cthaling/8003868/src/share/vm/shark/llvmHeaders.hpp:39,
> > > >>>> from /home/cthaling/8003868/src/share/vm/shark/sharkEntry.hpp:29,
> > > >>>> from /home/cthaling/8003868/src/share/vm/compiler/disassembler.cpp:51:
> > > >>>> /usr/local/include/llvm/Support/DataTypes.h:53:3: error: #error "Must #define __STDC_CONSTANT_MACROS before " "#including Support/DataTypes.h"
> > > >>>>
> > > >>>> and:
> > > >>>>
> > > >>>> In file included from /usr/local/include/llvm/Attributes.h:18,
> > > >>>> from /usr/local/include/llvm/Argument.h:18,
> > > >>>> from /home/cthaling/8003868/src/share/vm/shark/llvmHeaders.hpp:39,
> > > >>>> from /home/cthaling/8003868/src/share/vm/shark/sharkEntry.hpp:29,
> > > >>>> from /home/cthaling/8003868/src/share/vm/compiler/disassembler.cpp:51:
> > > >>>> /usr/local/include/llvm/Support/MathExtras.h: In function ‘bool llvm::isInt(int64_t)’:
> > > >>>> /usr/local/include/llvm/Support/MathExtras.h:38: error: there are no arguments to ‘INT64_C’ that depend on a template parameter, so a declaration of ‘INT64_C’ must be available
> > > >>>> /usr/local/include/llvm/Support/MathExtras.h:38: note: (if you use ‘-fpermissive’, G++ will accept your code, but allowing the use of an undeclared name is deprecated)
> > > >>>> /usr/local/include/llvm/Support/MathExtras.h:38: error: there are no arguments to ‘INT64_C’ that depend on a template parameter, so a declaration of ‘INT64_C’ must be available
> > > >>>> /usr/local/include/llvm/Support/MathExtras.h: In function ‘bool llvm::isUInt(uint64_t)’:
> > > >>>> /usr/local/include/llvm/Support/MathExtras.h:64: error: there are no arguments to ‘UINT64_C’ that depend on a template parameter, so a declaration of ‘UINT64_C’ must be available
> > > >>>> /usr/local/include/llvm/Support/MathExtras.h: In function ‘bool llvm::isIntN(unsigned int, int64_t)’:
> > > >>>> /usr/local/include/llvm/Support/MathExtras.h:96: error: ‘INT64_C’ was not declared in this scope
> > > >>>>
> > > >>>> Not sure if the latter is because of the former one. Have you seen this before?
> > > >>>
> > > >>> Yes, it's caused by the former. And yes, I have seen it before. IIRC,
> > > >>> this happens when certain cflags are not set correctly. The script
> > > >>> jdk/make/jdk_generic_profile.sh will call llvm-config to figure out the
> > > >>> correct flags. In order for this to work, you need to have the full path
> > > >>> to the llvm-config script set in the LLVM_CONFIG env variable. Were you
> > > >>> using the build script that I provided?
> > > >>
> > > >> No. I took your script and got the important environment variables. But I missed the LLVM_* ones. Usually we only build hotspot so we don't have this jdk script.
> > > >>
> > > >> Now that I have the LLVM_* variables it's even worse:
> > > >>
> > > >> /home/cthaling/8003868/src/share/vm/oops/oop.hpp:72: error: cast from type ‘markOopDesc* const volatile*’ to type ‘markOopDesc**’ casts away constness
> > > >>
> > > >> It's probably this guy: -Wcast-qual
> > > >
> > > > Got it:
> > > >
> > > > $ java -version
> > > > java version "1.8.0-ea"
> > > > Java(TM) SE Runtime Environment (build 1.8.0-ea-b65)
> > > > OpenJDK 64-Bit Shark VM (build 25.0-b11-internal-jvmg, mixed mode)
> > >
> > > I ran the compiler regression tests and Shark crashes in 5091921:
> > >
> > > cthaling at intelsdv03.us.oracle.com:~/8003868/test$ jtreg -workDir:$EXPORTHOME/jtreg -reportDir:$EXPORTHOME/jtreg -testjdk:$JAVA_HOME -verbose:summary compiler/5091921/
> > > Directory "/export/twisti/jtreg/scratch" not found: creating
> > > Passed: compiler/5091921/Test5091921.java
> > > Passed: compiler/5091921/Test6186134.java
> > > Passed: compiler/5091921/Test6196102.java
> > > Passed: compiler/5091921/Test6357214.java
> > > Passed: compiler/5091921/Test6559156.java
> > > Passed: compiler/5091921/Test6753639.java
> > > Passed: compiler/5091921/Test6850611.java
> > > Passed: compiler/5091921/Test6890943.java
> > > Passed: compiler/5091921/Test6897150.java
> > > Passed: compiler/5091921/Test6905845.java
> > > Passed: compiler/5091921/Test6931567.java
> > > /net/sqenfs-1.us.oracle.com/export1/comp/vm/tool/jtreg/execution/linux/bin/jtreg: line 139: 27784 Segmentation fault (core dumped) "${JT_JAVA}" $javaOpts -Dprogram=`basename "$0"` -jar "${JT_HOME}/lib/jtreg.jar" $jtregOpts
> > >
> > > You can also run all them with a simple make in test/ by setting:
> > >
> > > PRODUCT_HOME=$JAVA_HOME
> > > TESTDIRS=compiler
> > >
> >
> > This is interesting. I am not getting segfaults, but trip over
> > assertions inside LLVM (probably the same thingy). I tried looking (and
> > running through verifier) at the LLVM IR code, but nothing suspicious.
> > Then I turned off optimizations in LLVM and suddenly everything passed
> > (except for some tests that time out... probably performance tests that
> > time out because shark is too slow?). It could be a bug in LLVM, I
> > suspect this:
> >
> > http://llvm.org/bugs/show_bug.cgi?id=12470
> >
> > I am not sure though. I added a -XX:SharkOptimizationLevel switch to
> > select an optimization level at runtime. I will also check out the
> > LLVM-3.2 RC code and see if that's any better. I will keep you posted.
>
> So I tried with 3.2. Good news is that only minimal changes in Shark are
> required to do this, and another good news is that it does indeed make
> the crash go away :-) Will send out the patch later tonight (needs
> cleanup, and I need food ;-) ).
Ok, so here it comes:
http://cr.openjdk.java.net/~rkennke/shark/webrev.02/
The changes are (basically to enable support for LLVM3.2):
- Changed include for llvm/Support/IRBuilder.h > llvm/IRBuilder.h and
kept the old one in an #if SHARK_LLVM_VERSION <= 31 block.
- Added -XX:SharkOptimizationLevel=None|Less|Default|Aggressive option,
which is passed through to LLVM.
- Changed -XX:+VerifyFunctions to -XX:VerifyFunction=$PATTERN to
selectively verify functions.
- Changed SetCurrentDebugType -> setCurrentDebugType (and kept old one
in a version-check-block like above).
- Changed call to LLVM intrinsic llvm.memset.i32 to llvm.memset.p0i8.i32
(that has already been changed pre-3.2, but apparently the old one is
still working, but the verifier complains about it).
I ran all the tests in test/compiler and they all pass, except one that
timed out (dunno) and a few that failed with exceptions due to missing
JSR292 and one segfault (haven't looked yet).
Cheers,
Roman
More information about the hotspot-dev
mailing list