[x86_64 AVX2] weird crash due to RAX in String.compareTo(Object)

[Vladimir Ivanov]

Hi Liu,

I wasn't able to find any similar reports in the JBS.

> Forget about Test8005419.java because It can't reproduce this case neither.
> Has you seen this kind of crash before? May I file a new bug about it?
> I have already permutated all cases of two 70-length compareTo, but I still can't trigger it.
> What piece of information did I miss here?

The value in RAX is evidently broken.

str1 and str2 are already adjusted (+116 from 0th element) [1], so it 
seems the vector loop (COMPARE_WIDE_VECTORS_LOOP) has been run before 
and there was a mismatch encountered (VECTOR_NOT_EQUAL => 
COMPARE_16_CHARS). But then somehow there's no mismatch detected and it 
falls into vector loop once again.

The only plausible explanation I have is there's patching of String 
backing char array happening and it breaks the intrinsic which doesn't 
expect any concurrent modifications (and there shouldn't be any).

Best regards,
Vladimir Ivanov

RSI=0x00000000fe71487c is pointing into object: 0x00000000fe7147f8
[C
  - klass: {type array char}
  - length: 70
RDI=0x00000000fe711e44 is pointing into object: 0x00000000fe711dc0
[C
  - klass: {type array char}
  - length: 70

(lldb) p 0x00000000fe71487c - 0x00000000fe7147f8
(unsigned int) $0 = 132
(lldb) p 0x00000000fe711e44 - 0x00000000fe711dc0
(unsigned int) $2 = 132

> On 3/9/20, 1:00 AM, "hotspot-compiler-dev on behalf of Liu, Xin" <hotspot-compiler-dev-bounces at openjdk.java.net on behalf of xxinliu at amazon.com> wrote:
> 
>      Hi,
>      
>      I got some crash reports of C2 generated method String.compareTo(Object) on x86_64. This method is an intrinsics and defined in MacroAssembler::string_compare(macroAssembly_x86.cpp).
>      Yes, one interesting fact is the problem only happens on the bridge method compareTo(Object), deriving from the interface Comparable<String>.
>      
>      So far, I only see crashes in jdk8u because newer JDKs use AVX3 version by default, but I read the tip of jdk and AVX2 version is still the same. My concern is the bug is still there.  Have you seen this problem before?
>      
>      I found they all crash at an AVX instruction "0x00007ffb0d830235 vmovdqu ymm0, ymmword ptr [rdi + rax*2]", where RAX=0xffffffff00000036, RDI=0x00000000fe711e44.
>      JVM got SIGSEGV because of access violation. The faulty address is 0xfffffffefe711eb0, which is exactly (rax *2 + rdi).  It looks like result(rax)  has been overflowed. -4294967242
>      
>      AVX2 version comes from JDK-8005419. By changing the method signature a little bit in Test8005419.java, we can get String.compareTo(Object) AVX2 version as string_compare.S.
>      diff --git a/src/hotspot/test/compiler/8005419/Test8005419.java b/src/hotspot/test/compiler/8005419/Test8005419.java
>      index 201153e8a..1f8c57097 100644
>      --- a/src/hotspot/test/compiler/8005419/Test8005419.java
>      +++ b/src/hotspot/test/compiler/8005419/Test8005419.java
>      @@ -114,7 +114,7 @@ public class Test8005419 {
>               System.out.println("PASSED");
>           }
>      
>      -    private static int test(String str1, String str2) {
>      +    private static int test(Comparable<String>str1, String str2) {
>               return str1.compareTo(str2);
>           }
>       }
>      
>      Because it's an intrinsics, there's no code shape issue, right? I can't figure out how Rax becomes 0xffffffff00000036. I attached the original error message. According to RSI and RDI, the method was comparing two 70-length strings.
>      Test.java permutates all cases of two 70-length strings. Why I still can't hit this problem? Did I still miss anything?
>      
>      Thanks in advanced.
>      --lx
>      
>      
>      
>      
>