RFR: 8374516: -version asserts with "-XX:+UseAESCTRIntrinsics -XX:-UseAES": "need AES instructions and misaligned SSE support" in generate_counterMode_AESCrypt_Parallel()

[Shawn M Emery]

On Wed, 21 Jan 2026 08:32:59 GMT, Guanqiang Han <ghan at openjdk.org> wrote:

> Please review this change. Thanks!
> 
> **Description:**
> 
> VM crashes during startup on x86 when running with -XX:+UseAESCTRIntrinsics -XX:-UseAES. In this configuration, UseAESCTRIntrinsics may remain enabled while UseAES is explicitly disabled, and the VM generates AES-CTR stubs, hitting an assert(UseAES) in generate_counterMode_AESCrypt_Parallel().
> 
> **Fix:**
> 
> Update x86 flag initialization to enforce the dependency between UseAESCTRIntrinsics and UseAES. When UseAES is disabled, explicitly disable UseAESCTRIntrinsics (with a warning when it was set on the command line), aligning behavior with the existing UseAES/UseAESIntrinsics gating and avoiding stub generation with inconsistent flag states.
> 
> **Test:**
> 
> GHA

Nice work!  Just a couple of suggestions/comments.

src/hotspot/cpu/x86/vm_version_x86.cpp line 1141:

> 1139:       FLAG_SET_DEFAULT(UseAESIntrinsics, false);
> 1140:       if (UseAESCTRIntrinsics && !FLAG_IS_DEFAULT(UseAESCTRIntrinsics)) {
> 1141:         warning("AES_CTR intrinsics require UseAES flag to be enabled. Intrinsics will be disabled.");

I propose the following changes:
OLD
"Intrinsics will be disabled."
NEW
"AES_CTR intrinsics will be disabled."

test/hotspot/jtreg/compiler/cpuflags/TestUseAESCTRIntrinsicsWithUseAESDisabled.java line 28:

> 26:  * @bug 8374516
> 27:  * @summary Regression test for -XX:+UseAESCTRIntrinsics -XX:-UseAES crash
> 28:  * @requires os.arch=="amd64" | os.arch=="x86_64"

These are the only two architectures that exhibit this bug?
I was able to reproduce the problem with this test case on my x86_64 desktop and confirmed that the fix did indeed resolve the problem.

-------------

PR Review: https://git.openjdk.org/jdk/pull/29338#pullrequestreview-3743400759
PR Review Comment: https://git.openjdk.org/jdk/pull/29338#discussion_r2757536259
PR Review Comment: https://git.openjdk.org/jdk/pull/29338#discussion_r2757539553