RFR: 8324585: JVM native memory leak in PCKS11-NSS security provider [v3]

[Daniel Jeliński]

On Thu, 1 Feb 2024 08:33:13 GMT, Daniel Jeliński <djelinski at openjdk.org> wrote:

>> Please review this patch that fixes a memory leak in P11TlsPrfGenerator, which is triggered during TLS1.2 Finished message generation and verification.
>> 
>> The patch changes C_SignInit JNI method to free the mechanism data immediately after use. This matches the behavior of other Init methods (like C_EncryptInit). The patch also fixes a similar issue in other signature-related methods.
>> 
>> The change essentially reverts part of [JDK-8080462](https://bugs.openjdk.org/browse/JDK-8080462).
>> 
>> All sun/security/pkcs11 tests still pass with NSS ~3.35 and~ 3.91. All tier1-3 tests still pass.
>> 
>> EDIT:
>> Some sun/security/pkcs11 tests fail with NSS 3.64 and older, see [comment](https://github.com/openjdk/jdk/pull/17584#issuecomment-1914665234)
>
> Daniel Jeliński has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Improve wording

I applied your suggestion. Thanks for the review!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/17584#issuecomment-1920773619