RFR: 8325247: Memory leak in SessionKeyRef class def when using PKCS11 security provider [v2]

Daniel Jeliński djelinski at openjdk.org
Mon Feb 19 10:23:06 UTC 2024


> The reported leak was caused by the death of the `Cleanup-SunPKCS11` thread. The cleanup thread in turn died because of an exception thrown from `removeNativeKey` that resulted from 2 threads executing that method at the same time.
> 
> This PR adds a reachabilityFence to ensure that the key will only be enqueued for cleanup after the user thread is done with the `removeNativeKey` call.
> 
> No new regression test; the issue is extremely hard to reproduce in a reasonable time. Existing tier1-3 tests continue to pass.
> 
> In JBS I attached a PoC patch that changes the relative timing of operations; with that patch and without the changes from this PR I am able to reproduce the issue within a few seconds. With the changes from this PR the issue did not reproduce after 10 minutes of testing.

Daniel Jeliński has updated the pull request incrementally with one additional commit since the last revision:

  put reachabilityFence in a finally clause

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/17870/files
  - new: https://git.openjdk.org/jdk/pull/17870/files/8c0ed164..847af1bf

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=17870&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17870&range=00-01

  Stats: 6 lines in 1 file changed: 3 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/17870.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17870/head:pull/17870

PR: https://git.openjdk.org/jdk/pull/17870



More information about the security-dev mailing list