[x86_64 AVX2] weird crash due to RAX in String.compareTo(Object)

Liu, Xin xxinliu at amazon.com
Mon Mar 9 07:48:00 UTC 2020


Hi, 

I got some crash reports of C2 generated method String.compareTo(Object) on x86_64. This method is an intrinsics and defined in MacroAssembler::string_compare(macroAssembly_x86.cpp). 
Yes, one interesting fact is the problem only happens on the bridge method compareTo(Object), deriving from the interface Comparable<String>.

So far, I only see crashes in jdk8u because newer JDKs use AVX3 version by default, but I read the tip of jdk and AVX2 version is still the same. My concern is the bug is still there.  Have you seen this problem before? 

I found they all crash at an AVX instruction "0x00007ffb0d830235 vmovdqu ymm0, ymmword ptr [rdi + rax*2]", where RAX=0xffffffff00000036, RDI=0x00000000fe711e44.
JVM got SIGSEGV because of access violation. The faulty address is 0xfffffffefe711eb0, which is exactly (rax *2 + rdi).  It looks like result(rax)  has been overflowed. -4294967242

AVX2 version comes from JDK-8005419. By changing the method signature a little bit in Test8005419.java, we can get String.compareTo(Object) AVX2 version as string_compare.S.  
diff --git a/src/hotspot/test/compiler/8005419/Test8005419.java b/src/hotspot/test/compiler/8005419/Test8005419.java
index 201153e8a..1f8c57097 100644
--- a/src/hotspot/test/compiler/8005419/Test8005419.java
+++ b/src/hotspot/test/compiler/8005419/Test8005419.java
@@ -114,7 +114,7 @@ public class Test8005419 {
         System.out.println("PASSED");
     }

-    private static int test(String str1, String str2) {
+    private static int test(Comparable<String>str1, String str2) {
         return str1.compareTo(str2);
     }
 }

Because it's an intrinsics, there's no code shape issue, right? I can't figure out how Rax becomes 0xffffffff00000036. I attached the original error message. According to RSI and RDI, the method was comparing two 70-length strings. 
Test.java permutates all cases of two 70-length strings. Why I still can't hit this problem? Did I still miss anything?

Thanks in advanced. 
--lx





More information about the hotspot-compiler-dev mailing list