RFR: 8341527: AVX-512 intrinsic for SHA3 [v4]

Christian Hagedorn chagedorn at openjdk.org
Wed Oct 16 15:18:20 UTC 2024 

On Sat, 5 Oct 2024 16:47:12 GMT, Ferenc Rakoczi <duke at openjdk.org> wrote:

>> There is already an intrinsic for SHA-3 for aarch64, which gives significant speed improvement on that architecture, so this pull request is bringing similar improvement for tha x64 family of systems that have the AVX-512 extension. Rudimentary measurements show that 30-40% speed improvement can be achieved.
>
> Ferenc Rakoczi has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:
> 
>  - Merge branch 'master' into sha3-avx512-intrinsic
>  - fix windows build
>  - fix debug build
>  - 8341527: AVX-512 intrinsic for SHA3

This is not a review but I've run some testing with the current patch and found the following two failures on linux-x64-debug:

#### Failure 1
Tests: 
- `testlibrary_tests/ir_framework/tests/TestCPUFeatureCheck.java`
  - Additional flags: `-XX:+UseParallelGC -XX:+UseNUMA`
- `compiler/loopopts/superword/TestDependencyOffsets.java`
  - Additional flags: `-ea -esa -XX:CompileThreshold=100 -XX:+UnlockExperimentalVMOptions -server -XX:+TieredCompilation`
```	
CPU: total 12 (initial active 12) (6 cores per cpu, 2 threads per core) family 6 model 106 stepping 6 microcode 0x1, cx8, cmov, fxsr, ht, mmx, 3dnowpref, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, lzcnt, tsc, avx, avx2, aes, erms, clmul, bmi1, bmi2, adx, avx512f, avx512cd, sha, fma, clflush, hv, rdtscp, rdpid, fsrm, f16c, pku, ospke
CPU Model and flags from /proc/cpuinfo:
model name	: Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves nt_good wbnoinvd arat vnmi avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq la57 rdpid fsrm md_clear arch_capabilities


Failure:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (/opt/mach5/mesos/work_dir/slaves/a4a7850a-7c35-410a-b879-d77fbb2f6087-S144302/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/d6cec7c3-7401-41e9-aaad-f45b38c7a9e7/runs/9e85fc0d-9d6b-426f-b5d8-e84e2daa4c8c/workspace/open/src/hotspot/cpu/x86/assembler_x86.cpp:2979), pid=1550324, tid=1550336
#  Error: assert(VM_Version::supports_avx512dq()) failed
.....
Command Line: -Djava.library.path=/opt/mach5/mesos/work_dir/jib-master/install/2024-10-15-1659164.christian.hagedorn.jdk-test/linux-x64-debug.test/hotspot/jtreg/native -Xbootclasspath/a:. -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI -XX:MaxRAMPercentage=4.16667 -Dtest.boot.jdk=/opt/mach5/mesos/work_dir/jib-master/install/jdk/23/37/bundles/linux-x64/jdk-23_linux-x64_bin.tar.gz/jdk-23 -Djava.io.tmpdir=/opt/mach5/mesos/work_dir/slaves/a4a7850a-7c35-410a-b879-d77fbb2f6087-S151463/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/b87f227c-12f2-4145-8d72-0ba96c4ef814/runs/6cf6b0a6-7bb2-4e1d-97a8-2ad532639bfd/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_misc/tmp -XX:+UseParallelGC -XX:+UseNUMA -Dir.framework.server.port=42709 -XX:+UseKNLSetting -XX:-BackgroundCompilation -XX:CompileCommand=quiet compiler.lib.ir_framework.test.TestVM ir_framework.tests.TestCPUFeatureCheck
.....
Stack: [0x00007f4c047e9000,0x00007f4c048e9000],  sp=0x00007f4c048e57e0,  free space=1009k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x63bd61]  Assembler::kmovbl(KRegister, Register)+0x181  (assembler_x86.cpp:2979)
V  [libjvm.so+0x17615d4]  StubGenerator::generate_sha3_implCompress(bool, char const*)+0x274  (stubGenerator_x86_64_sha3.cpp:141)
V  [libjvm.so+0x176339a]  StubGenerator::generate_sha3_stubs()+0x3a  (stubGenerator_x86_64_sha3.cpp:86)
V  [libjvm.so+0x1711217]  StubGenerator::generate_compiler_stubs()+0x387  (stubGenerator_x86_64.cpp:4033)
V  [libjvm.so+0x1712ff8]  StubGenerator_generate(CodeBuffer*, StubCodeGenerator::StubsKind)+0xd8  (stubGenerator_x86_64.cpp:4242)
V  [libjvm.so+0x176d5af]  initialize_stubs(StubCodeGenerator::StubsKind, int, int, char const*, char const*, char const*)+0xef  (stubRoutines.cpp:245)
V  [libjvm.so+0x176f791]  compiler_stubs_init(bool)+0xa1  (stubRoutines.cpp:282)
V  [libjvm.so+0x87e5ae]  C2Compiler::init_c2_runtime()+0xbe  (c2compiler.cpp:92)
V  [libjvm.so+0x87e795]  C2Compiler::initialize()+0x35  (c2compiler.cpp:112)
V  [libjvm.so+0xa39716]  CompileBroker::init_compiler_runtime()+0xd6  (compileBroker.cpp:1771)
V  [libjvm.so+0xa3ff01]  CompileBroker::compiler_thread_loop()+0x121  (compileBroker.cpp:1913)
V  [libjvm.so+0xef158c]  JavaThread::thread_main_inner()+0xcc  (javaThread.cpp:759)
V  [libjvm.so+0x18199f6]  Thread::call_run()+0xb6  (thread.cpp:234)
V  [libjvm.so+0x14fc288]  thread_native_entry(Thread*)+0x128  (os_linux.cpp:858)


#### Failure 2
- `compiler/intrinsics/sha/cli/TestUseSHA3IntrinsicsOptionOnUnsupportedCPU.java`
  - Additional flags: `-server -Xmixed`
  
Output:

stderr: [Java HotSpot(TM) 64-Bit Server VM warning: SHA3 intrinsics require AVX512 instructions
java version "24-internal" 2025-03-18
Java(TM) SE Runtime Environment (fastdebug build 24-internal-2024-10-15-1659164.christian.hagedorn.jdk-test)
Java HotSpot(TM) 64-Bit Server VM (fastdebug build 24-internal-2024-10-15-1659164.christian.hagedorn.jdk-test, mixed mode, sharing)
]
 exitValue = 0

java.lang.AssertionError: Expected message not found: 'Intrinsics for SHA3-224, SHA3-256, SHA3-384 and SHA3-512 crypto hash functions not available on this CPU.'.
JVM should start with '-XX:-UseSHA3Intrinsics' flag without any warnings
	at jdk.test.lib.cli.CommandLineOptionTest.verifyOutput(CommandLineOptionTest.java:159)
	at jdk.test.lib.cli.CommandLineOptionTest.verifyJVMStartup(CommandLineOptionTest.java:130)
	at jdk.test.lib.cli.CommandLineOptionTest.verifySameJVMStartup(CommandLineOptionTest.java:209)
	at compiler.intrinsics.sha.cli.testcases.GenericTestCaseForUnsupportedX86CPU.verifyWarnings(GenericTestCaseForUnsupportedX86CPU.java:70)
	at compiler.intrinsics.sha.cli.DigestOptionsBase$TestCase.test(DigestOptionsBase.java:162)
	at compiler.intrinsics.sha.cli.DigestOptionsBase.runTestCases(DigestOptionsBase.java:139)
	at jdk.test.lib.cli.CommandLineOptionTest.test(CommandLineOptionTest.java:537)
	at compiler.intrinsics.sha.cli.TestUseSHA3IntrinsicsOptionOnUnsupportedCPU.main(TestUseSHA3IntrinsicsOptionOnUnsupportedCPU.java:56)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:573)
	at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138)
	at java.base/java.lang.Thread.run(Thread.java:1576)
Caused by: java.lang.RuntimeException: 'Intrinsics for SHA3-224, SHA3-256, SHA3-384 and SHA3-512 crypto hash functions not available on this CPU.' missing from stdout/stderr
	at jdk.test.lib.process.OutputAnalyzer.shouldMatch(OutputAnalyzer.java:372)
	at jdk.test.lib.cli.CommandLineOptionTest.verifyOutput(CommandLineOptionTest.java:154)
	... 11 more

JavaTest Message: Test threw exception: java.lang.AssertionError: Expected message not found: 'Intrinsics for SHA3-224, SHA3-256, SHA3-384 and SHA3-512 crypto hash functions not available on this CPU.'.
JVM should start with '-XX:-UseSHA3Intrinsics' flag without any warnings
JavaTest Message: shutting down test

STATUS:Failed.`main' threw exception: java.lang.AssertionError: Expected message not found: 'Intrinsics for SHA3-224, SHA3-256, SHA3-384 and SHA3-512 crypto hash functions not available on this CPU.'. JVM should start with '-XX:-UseSHA3Intrinsics' flag without any warnings

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21352#issuecomment-2417134494

More information about the hotspot-dev mailing list