RFR: 8259886 : Improve SSL session cache performance and scalability [v2]

[djelinski]

On Mon, 22 Feb 2021 21:31:21 GMT, Xue-Lei Andrew Fan <xuelei at openjdk.org> wrote:

>> Actually there's a much easier solution to reduce the number of slow `put()`s without making any behavioral changes.
>> The cache object could store the earliest expire time, and then exit `expungeExpiredEntries()` early when current time is earlier than the earliest expire time - when it is, we know that there are no expired items in the queue and we can skip the scan entirely.
>> @XueleiFan do you think the above is worth exploring?
>
>> Actually there's a much easier solution to reduce the number of slow `put()`s without making any behavioral changes.
>> The cache object could store the earliest expire time, and then exit `expungeExpiredEntries()` early when current time is earlier than the earliest expire time - when it is, we know that there are no expired items in the queue and we can skip the scan entirely.
>> @XueleiFan do you think the above is worth exploring?
> 
> Definitely, I think it is a good improvement.  Actually, it is a surprise to me that the current code is not working this way.
> 
> Sorry, I was/am on vacation, and the review could be delayed for a few days.

I reverted all earlier Cache changes, and added a new commit that caches the earliest expire time of all cached items. The observable behavior of the new code is identical to original - items are removed from cache at exactly the same time as before; we only skip scanning the cache when we know that there are no expired items inside.

The performance is substantially improved. There can be at most `cache size` scans in every `cache lifetime` period, which is roughly one scan every 4 seconds with the default SSL session cache settings. This is much better than possibly scanning on every `put()` that was possible before the changes.

My reduced set of benchmarks produced the following values:
Benchmark       (size)  (timeout)  Mode  Cnt    Score   Error  Units
CacheBench.put   20480      86400  avgt   25  148.345 ? 1.970  ns/op
CacheBench.put   20480          0  avgt   25  108.598 ? 3.787  ns/op
CacheBench.put  204800      86400  avgt   25  151.318 ? 1.872  ns/op
CacheBench.put  204800          0  avgt   25  106.650 ? 1.080  ns/op 
which is comparable to what was observed with the previous commits.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2255