Java 8 - pkcs11 threads hanging

[Andra Bennett]

Hello,

I am looking for some guidance on explaining a JDK 8 upgrade fixing a
troubling pkcs11-related behavior in our application.
I apologize for verbosity, any insight would be much appreciated.

Our application has upgraded from Open JDK 8 191 to JDK 8 232 and one of
the issues we were previously seeing - namely threads locking up in our
application when creating message signatures in a multithreaded environment
using Sun PKCS11 provider - has been mitigated.

We know javax.crypto.Mac is no thread safe so we always created a new
instance. However, at a certain thread count our application would just
freeze (easily reproducible in a more focused unit test).

We would like to see if there was a set of changes that would have targeted
pkcs11 threads hanging on PKCS11 create/destroy object threads, e.g.

sun.security.pkcs11.wrapper.PKCS11.C_DestroyObject(Native Method)
    sun.security.pkcs11.SessionKeyRef.dispose(P11Key.java:1138)
    sun.security.pkcs11.SessionKeyRef.drainRefQueueBounded(P11Key.java:1114)
    sun.security.pkcs11.SessionKeyRef.<init>(P11Key.java:1129)
    sun.security.pkcs11.P11Key.<init>(P11Key.java:119)
    sun.security.pkcs11.P11Key$P11SecretKey.<init>(P11Key.java:405)
    sun.security.pkcs11.P11Key.secretKey(P11Key.java:292)
    sun.security.pkcs11.P11SecretKeyFactory.createKey(P11SecretKeyFactory.java:267)
    sun.security.pkcs11.P11SecretKeyFactory.convertKey(P11SecretKeyFactory.java:175)
    sun.security.pkcs11.P11SecretKeyFactory.convertKey(P11SecretKeyFactory.java:111)
    sun.security.pkcs11.P11Mac.engineInit(P11Mac.java:206)
    javax.crypto.Mac.chooseProvider(Mac.java:350)


We searched the Open JDK JIRAs at
https://bugs.openjdk.java.net/projects/JDK/issues with filter "project =
JDK and fixVersion ~ "8u*" and summary ~ "pkcs11"" but could not find any
relevant issues being addressed. Also searched for javax.crypto changes for
8u versions but even less changes were related to that.

We haven't identified an actual JDK bug filed with this issue, except a
RedHat bug originating the pkcs11 execution in an SSL handshake:
https://access.redhat.com/solutions/312503


Thank you in advance,
Andra