[16] RFR(S): 8249608: Vector register used by C2 compiled method corrupted at safepoint

[Tobias Hartmann]

Hi,

please review the following fix:
https://bugs.openjdk.java.net/browse/JDK-8249608
http://cr.openjdk.java.net/~thartmann/8249608/webrev.00/

The problem is very similar to JDK-8193518 [1], a vector register (ymm0) used for vectorization of a
loop in a C2 compiled method is corrupted at a safepoint. Again, the root cause is the superword
optimization setting 'max_vector_size' to 16 bytes instead of 32 bytes which leads to the nmethod
being marked as !has_wide_vectors and the safepoint handler not saving vector registers [3].

This time, the problem is that the superword code only updates 'max_vlen_in_bytes' if 'vlen >
max_vlen'. In the failing case, 'vlen' is 4 for all packs (see [4]) but 'vlen_in_bytes' is 16 for
the 4 x int StoreVector and 32 for the 4 x long StoreVector. Once we've processed the int
StoreVector, we are not updating 'max_vlen_in_bytes' when processing long StoreVector because 'vlen'
is equal.

The fix is to make sure to always update 'max_vlen_in_bytes'.

When looking at JDK-8193518 [1], I've noticed that the corresponding regression test was never
pushed. I've added it to this webrev and extended it such that it also covers the new issue.

Thanks,
Tobias

[1] https://bugs.openjdk.java.net/browse/JDK-8193518
[2] http://hg.openjdk.java.net/jdk/jdk/file/1f74c0319302/src/hotspot/share/opto/output.cpp#l3313
[3]
http://hg.openjdk.java.net/jdk/jdk/file/1f74c0319302/src/hotspot/share/runtime/sharedRuntime.cpp#l551

[4] -XX:+TraceSuperWord output:

After filter_packs
packset
Pack: 0
 align: 0 	 1101	StoreL	===  1115  1120  1102  174  [[ 1098 ]]  @long[int:>=0]:exact+any *, idx=6;
Memory: @long[int:>=0]:NotNull:exact+any *, idx=6; !orig=993,214,[1012] !jvms: Test::test @ bci:17
Test::main @ bci:8
 align: 8 	 1098	StoreL	===  1115  1101  1099  174  [[ 993 ]]  @long[int:>=0]:exact+any *, idx=6;
Memory: @long[int:>=0]:NotNull:exact+any *, idx=6; !orig=214,[1012] !jvms: Test::test @ bci:17
Test::main @ bci:8
 align: 16 	 993	StoreL	===  1115  1098  994  174  [[ 866  214 ]]  @long[int:>=0]:exact+any *,
idx=6;  Memory: @long[int:>=0]:NotNull:exact+any *, idx=6; !orig=214,[1012] !jvms: Test::test @
bci:17 Test::main @ bci:8
 align: 24 	 214	StoreL	===  1115  993  212  174  [[ 1120  864  255 ]]  @long[int:>=0]:exact+any *,
idx=6;  Memory: @long[int:>=0]:NotNull:exact+any *, idx=6; !orig=[1012] !jvms: Test::test @ bci:17
Test::main @ bci:8
Pack: 1
 align: 0 	 1097	StoreI	===  1115  1119  1106  41  [[ 1096 ]]  @int[int:>=0]:exact+any *, idx=8;
Memory: @int[int:>=0]:NotNull:exact+any *, idx=8; !orig=989,253,[1009] !jvms: Test::test @ bci:23
Test::main @ bci:8
 align: 4 	 1096	StoreI	===  1115  1097  1104  41  [[ 989 ]]  @int[int:>=0]:exact+any *, idx=8;
Memory: @int[int:>=0]:NotNull:exact+any *, idx=8; !orig=253,[1009] !jvms: Test::test @ bci:23
Test::main @ bci:8
 align: 8 	 989	StoreI	===  1115  1096  996  41  [[ 867  253 ]]  @int[int:>=0]:exact+any *, idx=8;
Memory: @int[int:>=0]:NotNull:exact+any *, idx=8; !orig=253,[1009] !jvms: Test::test @ bci:23
Test::main @ bci:8
 align: 12 	 253	StoreI	===  1115  989  251  41  [[ 1119  860  255 ]]  @int[int:>=0]:exact+any *,
idx=8;  Memory: @int[int:>=0]:NotNull:exact+any *, idx=8; !orig=[1009] !jvms: Test::test @ bci:23
Test::main @ bci:8

new Vector node:  1491	ReplicateI	=== _  41  [[]]  #vectorx[4]:{int}
new Vector node:  1492	StoreVector	===  1115  1119  1106  1491  [[ 1487  1119  255  1486 ]]
@int[int:>=0]:NotNull:exact+any *, idx=8; mismatched  Memory: @int[int:>=0]:NotNull:exact+any *,
idx=8; !orig=[1097],[989],[253],[1009] !jvms: Test::test @ bci:23 Test::main @ bci:8
new Vector node:  1493	ReplicateL	=== _  174  [[]]  #vectory[4]:{long}
new Vector node:  1494	StoreVector	===  1115  1120  1102  1493  [[ 1489  1120  255  1488 ]]
@long[int:>=0]:NotNull:exact+any *, idx=6; mismatched  Memory: @long[int:>=0]:NotNull:exact+any *,
idx=6; !orig=[1101],[993],[214],[1012] !jvms: Test::test @ bci:17 Test::main @ bci:8