RFR: 8307185: pkcs11 native libraries make JNI calls into java code while holding GC lock

Daniel Jeliński djelinski at openjdk.org
Wed Jul 19 12:35:09 UTC 2023


This patch fixes random deadlocks in PKCS11 decryption and encryption code.

The deadlocks were caused by object allocation in `ckAssertReturnValueOK` waiting for GC; GC was in turn waiting for `ReleasePrimitiveArrayCritical`, which never happened.

The fix moves the call to `ckAssertReturnValueOK` after `ReleasePrimitiveArrayCritical`.

All tests in sun/security/pkcs11 were executed with `JTREG=JAVA_OPTIONS=-Xcheck:jni` with and without this patch.
Without this patch, 5 tests produced the following warning:

Warning: Calling other JNI functions in the scope of Get/ReleasePrimitiveArrayCritical or Get/ReleaseStringCritical

With this patch the warning was not observed.

Tier2-3 tests are still green.

-------------

Commit messages:
 - Fix JNI usage in PKCS11

Changes: https://git.openjdk.org/jdk/pull/14931/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14931&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8307185
  Stats: 34 lines in 2 files changed: 16 ins; 11 del; 7 mod
  Patch: https://git.openjdk.org/jdk/pull/14931.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/14931/head:pull/14931

PR: https://git.openjdk.org/jdk/pull/14931


More information about the security-dev mailing list