RFR (L): 8003868: fix shark for latest HotSpot and LLVM [Was: Re: RFR: Fix shark for latest Hotspot and LLVM]
Christian Thalinger
christian.thalinger at oracle.com
Tue Nov 27 10:59:47 PST 2012
On Nov 27, 2012, at 9:24 AM, Roman Kennke <rkennke at redhat.com> wrote:
> Am Montag, den 26.11.2012, 17:14 -0800 schrieb Christian Thalinger:
>> On Nov 26, 2012, at 4:44 PM, Roman Kennke <rkennke at redhat.com> wrote:
>>
>>> Am Montag, den 26.11.2012, 15:43 -0800 schrieb Christian Thalinger:
>>>> On Nov 26, 2012, at 3:18 PM, Roman Kennke <rkennke at redhat.com> wrote:
>>>>
>>>>> Am Montag, den 26.11.2012, 11:55 -0800 schrieb Christian Thalinger:
>>>>>> On Nov 23, 2012, at 9:14 AM, Roman Kennke <rkennke at redhat.com> wrote:
>>>>>>
>>>>>>> Am Mittwoch, den 21.11.2012, 14:52 -0800 schrieb Christian Thalinger:
>>>>>>>> On Nov 21, 2012, at 2:22 PM, Christian Thalinger <christian.thalinger at oracle.com> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Nov 21, 2012, at 2:17 PM, Christian Thalinger <christian.thalinger at oracle.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Nov 21, 2012, at 12:54 PM, Roman Kennke <rkennke at redhat.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Am Mittwoch, den 21.11.2012, 12:47 -0800 schrieb Christian Thalinger:
>>>>>>>>>>>> On Nov 21, 2012, at 11:43 AM, Christian Thalinger <christian.thalinger at oracle.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> On Nov 21, 2012, at 9:31 AM, Roman Kennke <rkennke at redhat.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi there,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> during the last days I worked on fixing the Shark compiler for Hotspot
>>>>>>>>>>>>>> to get it to build and run again, with the latest Hotspot code and LLVM.
>>>>>>>>>>>>>> Here are some details:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - A lot of changes are just to make it build and the compiler happy. For
>>>>>>>>>>>>>> example, I had to remove a lot of 'const' qualifiers because of API
>>>>>>>>>>>>>> changes in LLVM.
>>>>>>>>>>>>>> - Most other changes have to do with the split of the oop and metadata
>>>>>>>>>>>>>> class hierarchies in Hotspot.
>>>>>>>>>>>>>> - Then there have been a few changes caused by LLVM changes and
>>>>>>>>>>>>>> improvements, most notably the LLVM intrinsics for atomic operations
>>>>>>>>>>>>>> (memory barrier and cmpxchg) have been removed and now have a
>>>>>>>>>>>>>> representation directly in LLVM's IR. This makes our code a little
>>>>>>>>>>>>>> nicer.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I tested this by running a number of applications, most notably Eclipse
>>>>>>>>>>>>>> (which is notoriously difficult on VMs), Java2Demo, SwingSet2 and a
>>>>>>>>>>>>>> bunch of other stuff.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I would like to get this integrated into OpenJDK now if possible. You
>>>>>>>>>>>>>> can find the full webrev here:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/shark/webrev.00/
>>>>>>>>>>>>>
>>>>>>>>>>>>> The changes seem to touch almost only shark files so these should be fine. One question though:
>>>>>>>>>>>>>
>>>>>>>>>>>>> + develop(bool, SharkShowCompiledMethods, false, \
>>>>>>>>>>>>>
>>>>>>>>>>>>> Isn't PrintCompilation doing that already?
>>>>>>>>>>>>>
>>>>>>>>>>>>> The shared code changes look good. I filed:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 8003868: fix shark for latest HotSpot and LLVM
>>>>>>>>>>>>>
>>>>>>>>>>>>> -- Chris
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There are also a very minor change required in JDK:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/shark/webrev-jdk-00/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In order to build it, apply the patches on hsx/hotspot-comp 's hotspot
>>>>>>>>>>>>>> and jdk repositories respectivly. Find my build script here:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://cr.openjdk.java.net/~rkennke/shark/Build8-zero-shark
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (Review and adjust variables to your settings, most notably you will
>>>>>>>>>>>>>> need to change LLVM_CONFIG to point to your LLVM 3.1 installation.)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please let me know if there are any issues or how we can get this
>>>>>>>>>>>>>> integrated into Hotspot.
>>>>>>>>>>>>
>>>>>>>>>>>> Finally I installed LLVM on one of our machines to be able to do a Shark build once in a while. When I try to do a jvmgshark build I get:
>>>>>>>>>>>>
>>>>>>>>>>>> In file included from /usr/local/include/llvm/Support/PointerLikeTypeTraits.h:18,
>>>>>>>>>>>> from /usr/local/include/llvm/ADT/PointerIntPair.h:17,
>>>>>>>>>>>> from /usr/local/include/llvm/Use.h:28,
>>>>>>>>>>>> from /usr/local/include/llvm/Value.h:17,
>>>>>>>>>>>> from /usr/local/include/llvm/Argument.h:17,
>>>>>>>>>>>> from /home/cthaling/8003868/src/share/vm/shark/llvmHeaders.hpp:39,
>>>>>>>>>>>> from /home/cthaling/8003868/src/share/vm/shark/sharkEntry.hpp:29,
>>>>>>>>>>>> from /home/cthaling/8003868/src/share/vm/compiler/disassembler.cpp:51:
>>>>>>>>>>>> /usr/local/include/llvm/Support/DataTypes.h:53:3: error: #error "Must #define __STDC_CONSTANT_MACROS before " "#including Support/DataTypes.h"
>>>>>>>>>>>>
>>>>>>>>>>>> and:
>>>>>>>>>>>>
>>>>>>>>>>>> In file included from /usr/local/include/llvm/Attributes.h:18,
>>>>>>>>>>>> from /usr/local/include/llvm/Argument.h:18,
>>>>>>>>>>>> from /home/cthaling/8003868/src/share/vm/shark/llvmHeaders.hpp:39,
>>>>>>>>>>>> from /home/cthaling/8003868/src/share/vm/shark/sharkEntry.hpp:29,
>>>>>>>>>>>> from /home/cthaling/8003868/src/share/vm/compiler/disassembler.cpp:51:
>>>>>>>>>>>> /usr/local/include/llvm/Support/MathExtras.h: In function ‘bool llvm::isInt(int64_t)’:
>>>>>>>>>>>> /usr/local/include/llvm/Support/MathExtras.h:38: error: there are no arguments to ‘INT64_C’ that depend on a template parameter, so a declaration of ‘INT64_C’ must be available
>>>>>>>>>>>> /usr/local/include/llvm/Support/MathExtras.h:38: note: (if you use ‘-fpermissive’, G++ will accept your code, but allowing the use of an undeclared name is deprecated)
>>>>>>>>>>>> /usr/local/include/llvm/Support/MathExtras.h:38: error: there are no arguments to ‘INT64_C’ that depend on a template parameter, so a declaration of ‘INT64_C’ must be available
>>>>>>>>>>>> /usr/local/include/llvm/Support/MathExtras.h: In function ‘bool llvm::isUInt(uint64_t)’:
>>>>>>>>>>>> /usr/local/include/llvm/Support/MathExtras.h:64: error: there are no arguments to ‘UINT64_C’ that depend on a template parameter, so a declaration of ‘UINT64_C’ must be available
>>>>>>>>>>>> /usr/local/include/llvm/Support/MathExtras.h: In function ‘bool llvm::isIntN(unsigned int, int64_t)’:
>>>>>>>>>>>> /usr/local/include/llvm/Support/MathExtras.h:96: error: ‘INT64_C’ was not declared in this scope
>>>>>>>>>>>>
>>>>>>>>>>>> Not sure if the latter is because of the former one. Have you seen this before?
>>>>>>>>>>>
>>>>>>>>>>> Yes, it's caused by the former. And yes, I have seen it before. IIRC,
>>>>>>>>>>> this happens when certain cflags are not set correctly. The script
>>>>>>>>>>> jdk/make/jdk_generic_profile.sh will call llvm-config to figure out the
>>>>>>>>>>> correct flags. In order for this to work, you need to have the full path
>>>>>>>>>>> to the llvm-config script set in the LLVM_CONFIG env variable. Were you
>>>>>>>>>>> using the build script that I provided?
>>>>>>>>>>
>>>>>>>>>> No. I took your script and got the important environment variables. But I missed the LLVM_* ones. Usually we only build hotspot so we don't have this jdk script.
>>>>>>>>>>
>>>>>>>>>> Now that I have the LLVM_* variables it's even worse:
>>>>>>>>>>
>>>>>>>>>> /home/cthaling/8003868/src/share/vm/oops/oop.hpp:72: error: cast from type ‘markOopDesc* const volatile*’ to type ‘markOopDesc**’ casts away constness
>>>>>>>>>>
>>>>>>>>>> It's probably this guy: -Wcast-qual
>>>>>>>>>
>>>>>>>>> Got it:
>>>>>>>>>
>>>>>>>>> $ java -version
>>>>>>>>> java version "1.8.0-ea"
>>>>>>>>> Java(TM) SE Runtime Environment (build 1.8.0-ea-b65)
>>>>>>>>> OpenJDK 64-Bit Shark VM (build 25.0-b11-internal-jvmg, mixed mode)
>>>>>>>>
>>>>>>>> I ran the compiler regression tests and Shark crashes in 5091921:
>>>>>>>>
>>>>>>>> cthaling at intelsdv03.us.oracle.com:~/8003868/test$ jtreg -workDir:$EXPORTHOME/jtreg -reportDir:$EXPORTHOME/jtreg -testjdk:$JAVA_HOME -verbose:summary compiler/5091921/
>>>>>>>> Directory "/export/twisti/jtreg/scratch" not found: creating
>>>>>>>> Passed: compiler/5091921/Test5091921.java
>>>>>>>> Passed: compiler/5091921/Test6186134.java
>>>>>>>> Passed: compiler/5091921/Test6196102.java
>>>>>>>> Passed: compiler/5091921/Test6357214.java
>>>>>>>> Passed: compiler/5091921/Test6559156.java
>>>>>>>> Passed: compiler/5091921/Test6753639.java
>>>>>>>> Passed: compiler/5091921/Test6850611.java
>>>>>>>> Passed: compiler/5091921/Test6890943.java
>>>>>>>> Passed: compiler/5091921/Test6897150.java
>>>>>>>> Passed: compiler/5091921/Test6905845.java
>>>>>>>> Passed: compiler/5091921/Test6931567.java
>>>>>>>> /net/sqenfs-1.us.oracle.com/export1/comp/vm/tool/jtreg/execution/linux/bin/jtreg: line 139: 27784 Segmentation fault (core dumped) "${JT_JAVA}" $javaOpts -Dprogram=`basename "$0"` -jar "${JT_HOME}/lib/jtreg.jar" $jtregOpts
>>>>>>>>
>>>>>>>> You can also run all them with a simple make in test/ by setting:
>>>>>>>>
>>>>>>>> PRODUCT_HOME=$JAVA_HOME
>>>>>>>> TESTDIRS=compiler
>>>>>>>
>>>>>>> Alright, I found another fairly grave bug that I would like to include a
>>>>>>> fix for: apparently, a 'fence' is not enough of a memory barrier for
>>>>>>> volatile putfield/getfield (duh). Which basically broke all of j.u.c.
>>>>>>> LLVM offers atomic loads and stores, which seem to be exactly what is
>>>>>>> needed for volatile field access. However, it does not provide helper
>>>>>>> functions for those in llvm::IRBuilder so I wrote my own. This should be
>>>>>>> the final patch for now (unless you find something else).
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~rkennke/shark/webrev.03/
>>>>>>
>>>>>> Hmm. Maybe I did something wrong but I've already rebuilt twice:
>>>>>>
>>>>>> $ java -Xcomp -version
>>>>>> Value type size is target-dependent. Ask TLI.
>>>>>> UNREACHABLE executed at /usr/local/src/llvm-3.1.src/include/llvm/CodeGen/ValueTypes.h:257!
>>>>>> Stack dump:
>>>>>> 0. Running pass 'X86 DAG->DAG Instruction Selection' on function '@"java.lang.System::getProperty"'
>>>>>> Aborted (core dumped)
>>>>>
>>>>> Arg! The last couple of changes I did only with LLVM3.2, where the
>>>>> problem disappears. Apparently, LLVM3.1 (and pre) don't deal well with
>>>>> atomic load/store :-( I re-introduced the CreateMemoryBarrier call and
>>>>> use that for SHARK_LLVM_VERSION <= 31.
>>>>>
>>>>> http://cr.openjdk.java.net/~rkennke/shark/webrev.04/
>>>>>
>>>>> Hope that works better :-)
>>>>
>>>> I'm so sorry but...
>>>>
>>>> /export/twisti/build/8003868/build/linux_amd64_shark/product/libjvm.so: undefined reference to `SharkBuilder::memory_barrier()'
>>>
>>> Gaaa, what the... I thought I did clean rebuilds with both llvm3.2 and
>>> llvm3.1, but apparently not (maybe I shouldn't work after 1am). This
>>> (hopefully final final) patch re-instates the missing memory_barrier()
>>> method:
>>>
>>>
>>> http://cr.openjdk.java.net/~rkennke/shark/webrev.05/
>>>
>>> Sorry for the messy back-and-forth.
>>
>> Again, so sorry:
>>
>> $ java -Xcomp -version
>> LLVM ERROR: Program used external function 'llvm.memory.barrier' which could not be resolved!
>>
>> Send a new patch tomorrow after some sleep ;-)
>
> Yeah, apparently 'replaced by' means that the old thing (the intrinsics)
> are indeed gone ;-)
>
> The problem is that the correct way to implement it (atomic load/store)
> doesn't work, the 'old way' (the memory_barrier() intrinsic call)
> doesn't work either, I also tried CreateAtomicRMW() which is probably
> not 100% correct, but would have done the job, but that doesn't work
> either (it throws the same error as the atomic load/store ...). The
> problem seems to only appear on 64bit volatile access.
>
> I don't know a really good solution that doesn't require jumping through
> big hoops, and I don't feel like working around LLVM bugs like this,
> especially since LLVM 3.2 (which should be released real soon now) works
> just fine. If you have a suggestion, please let me know, otherwise I
> suggest the following patch, which gets rid of all the LLVM31 blocks
> again and creates atomic load/store instructions (and requires LLVM 3.2
> which can be found here
> http://llvm.org/svn/llvm-project/llvm/branches/release_32/ ).
>
> http://cr.openjdk.java.net/~rkennke/shark/webrev.06/
That's a reasonable thing to do given the tentative release date of December 16th. While running the compiler regression tests I got a couple of failures. You might want to address them with separate bugs.
-- Chris
--------------------------------------------------
TEST: compiler/6539464/Test.java
JDK under test: (/export/twisti/build/8003868/build/jdk-linux)
java version "1.8.0-ea"
Java(TM) SE Runtime Environment (build 1.8.0-ea-b65)
OpenJDK 64-Bit Shark VM (build 25.0-b11-internal, mixed mode)
ACTION: build -- Passed. Build successful
REASON: Named class compiled on demand
TIME: 1.988 seconds
messages:
command: build Test
reason: Named class compiled on demand
elapsed time (seconds): 1.988
ACTION: compile -- Passed. Compilation successful
REASON: .class file out of date or does not exist
TIME: 1.986 seconds
messages:
command: compile /home/cthaling/8003868/test/compiler/6539464/Test.java
reason: .class file out of date or does not exist
elapsed time (seconds): 1.986
STDOUT:
STDERR:
ACTION: main -- Failed. Execution failed: `main' threw exception: java.lang.InternalError: Math.log produces inconsistent results: 9.752490228984199 != 9.7524902289842
REASON: User specified action: run main/othervm -Xcomp -XX:CompileOnly=Test.main Test
TIME: 0.121 seconds
messages:
command: main -Xcomp -XX:CompileOnly=Test.mainTest
reason: User specified action: run main/othervm -Xcomp -XX:CompileOnly=Test.main Test
elapsed time (seconds): 0.121
STDOUT:
STDERR:
java.lang.InternalError: Math.log produces inconsistent results: 9.752490228984199 != 9.7524902289842
at Test.main(Test.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:474)
at com.sun.javatest.regtest.MainWrapper$MainThread.run(MainWrapper.java:94)
at java.lang.Thread.run(Thread.java:722)
JavaTest Message: Test threw exception: java.lang.InternalError: Math.log produces inconsistent results: 9.752490228984199 != 9.7524902289842
JavaTest Message: shutting down test
STATUS:Failed.`main' threw exception: java.lang.InternalError: Math.log produces inconsistent results: 9.752490228984199 != 9.7524902289842
TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.InternalError: Math.log produces inconsistent results: 9.752490228984199 != 9.7524902289842
--------------------------------------------------
--------------------------------------------------
TEST: compiler/6796786/Test6796786.java
JDK under test: (/export/twisti/build/8003868/build/jdk-linux)
java version "1.8.0-ea"
Java(TM) SE Runtime Environment (build 1.8.0-ea-b65)
OpenJDK 64-Bit Shark VM (build 25.0-b11-internal, mixed mode)
ACTION: build -- Passed. Build successful
REASON: Named class compiled on demand
TIME: 2.212 seconds
messages:
command: build Test6796786
reason: Named class compiled on demand
elapsed time (seconds): 2.212
ACTION: compile -- Passed. Compilation successful
REASON: .class file out of date or does not exist
TIME: 2.209 seconds
messages:
command: compile /home/cthaling/8003868/test/compiler/6796786/Test6796786.java
reason: .class file out of date or does not exist
elapsed time (seconds): 2.209
STDOUT:
STDERR:
ACTION: main -- Failed. Unexpected exit from test [exit code: 1]
REASON: User specified action: run main/othervm -Xbatch Test6796786
TIME: 0.638 seconds
messages:
command: main -XbatchTest6796786
reason: User specified action: run main/othervm -Xbatch Test6796786
elapsed time (seconds): 0.638
STDOUT:
STDERR:
LLVM ERROR: Cannot select: 0x2ab78c065100: f32,ch = AtomicLoad 0x2ab78c64dad0:1, 0x2ab78c069c40<Volatile LD4[%addr26](align=8)> [ORD=26732] [ID=49]
0x2ab78c069c40: i64 = add 0x2ab78c64dad0, 0x2ab78c64d9d0 [ORD=26730] [ID=48]
0x2ab78c64dad0: i64,ch = load 0x2ab78c061710:1, 0x2ab78c0649f0, 0x2ab78c64dcd0<LD8[%105]> [ORD=26729] [ID=47]
0x2ab78c0649f0: i64 = add 0x2ab78c069b40, 0x2ab78c0ef4b0 [ORD=26727] [ID=40]
0x2ab78c069b40: i64,ch = CopyFromReg 0x2ab75c0be7a0, 0x2ab78c061110 [ORD=26721] [ID=29]
0x2ab78c061110: i64 = Register %vreg31 [ORD=26721] [ID=2]
0x2ab78c0ef4b0: i64 = Constant<48> [ORD=26727] [ID=6]
0x2ab78c64dcd0: i64 = undef [ORD=26723] [ID=4]
0x2ab78c64d9d0: i64 = Constant<204> [ORD=26730] [ID=7]
In function: Test6796786::main
TEST RESULT: Failed. Unexpected exit from test [exit code: 1]
--------------------------------------------------
--------------------------------------------------
TEST: compiler/6932496/Test6932496.java
JDK under test: (/export/twisti/build/8003868/build/jdk-linux)
java version "1.8.0-ea"
Java(TM) SE Runtime Environment (build 1.8.0-ea-b65)
OpenJDK 64-Bit Shark VM (build 25.0-b11-internal, mixed mode)
ACTION: compile -- Passed. Compilation successful
REASON: User specified action: run compile -source 1.5 -target 1.5 -XDjsrlimit=0 Test6932496.java
TIME: 2.022 seconds
messages:
command: compile -source 1.5 -target 1.5 -XDjsrlimit=0 /home/cthaling/8003868/test/compiler/6932496/Test6932496.java
reason: User specified action: run compile -source 1.5 -target 1.5 -XDjsrlimit=0 Test6932496.java
elapsed time (seconds): 2.022
STDOUT:
STDERR:
warning: [options] bootstrap class path not set in conjunction with -source 1.5
1 warning
ACTION: build -- Passed. All files up to date
REASON: Named class compiled on demand
TIME: 0.002 seconds
messages:
command: build Test6932496
reason: Named class compiled on demand
elapsed time (seconds): 0.002
ACTION: main -- Failed. Unexpected exit from test [exit code: 134]
REASON: User specified action: run main/othervm -Xcomp -XX:CompileOnly=Test6932496.m Test6932496
TIME: 0.673 seconds
messages:
command: main -Xcomp -XX:CompileOnly=Test6932496.mTest6932496
reason: User specified action: run main/othervm -Xcomp -XX:CompileOnly=Test6932496.m Test6932496
elapsed time (seconds): 0.673
STDOUT:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# Internal Error (os_linux_zero.cpp:254), pid=6346, tid=47237501929232
# fatal error: caught unhandled signal 7
#
# JRE version: Java(TM) SE Runtime Environment (8.0-b65) (build 1.8.0-ea-b65)
# Java VM: OpenJDK 64-Bit Shark VM (25.0-b11-internal compiled mode linux-amd64 )
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /export/twisti/build/8003868/build/linux-x86_64/testoutput/JTwork/scratch/hs_err_pid6346.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.sun.com/bugreport/crash.jsp
#
STDERR:
TEST RESULT: Failed. Unexpected exit from test [exit code: 134]
--------------------------------------------------
More information about the hotspot-dev
mailing list