RFR: 8258833: Cancel multi-part cipher operations in SunPKCS11 after failures [v3]

[Valerie Peng]

On Fri, 8 Jan 2021 14:06:18 GMT, Martin Balao <mbalao at openjdk.org> wrote:

>> So, after this PKCS11 session failed with an error, you can continue using it, say give it another byte[] and it would continue to work as if the earlier failed call never happened? I have not really tried it and thought that this session is unusable due to the combination of an existing error (say some exception happened during the encrypt update call) and active operation state.
>
> Yes, that's right: you can use the session to continue the same operation. In fact, there is a pattern -also used in some Windows API functions- that you send an output buffer of length 0, you get a CKR_BUFFER_TOO_SMALL error and the buffer size that would have been required; and you make a second call with the right buffer size. This pattern is used from the OpenJDK native library that communicates with the PKCS#11 lib. Some types of errors (those that do not free the context) are not fatal. The problem is that we end up returning a session to the Session Manager and we try to use the session to initialize a different operation, while the previous is still active.

The double-call-query usage pattern is well documented in the PKCS#11 spec. So, there is no ambiguity there. As for what PKCS11 errors are fatal and the resulting session state seems to be a gray area. Anyway, I agree that defensive cancelling this to work with all PKCS11 library is good so to ensure subsequent operations using the reused session work.

I can't think of other better terms. Fine with me to just use "leak" w/ your analogy explanation.

-------------

PR: https://git.openjdk.java.net/jdk/pull/1901