Lots of allocations in CipherCore

[Sebastian Stenzel]

Hi Tony,

while running regression tests, I noticed that my assumption only holds true for unpadded ciphers. For padded ciphers, we still need the temporary buffer, as only the unpadded cleartext shall be written to the output buffer.

That said, here is my PR. I'd be happy if you could review it (and act as a sponsor for the commit): https://github.com/openjdk/jdk/pull/7230 <https://github.com/openjdk/jdk/pull/7230>

Cheers!
Sebastian

> On 25. Jan 2022, at 22:43, Anthony Scarpino <anthony.scarpino at oracle.com> wrote:
> 
> Hi,
> 
> I think it's a mistake. Looking at the old code I believe the if() for cipher.save() above that was suppose to include the new byte allocation and offset, but got missed.
> 
> Feel free to fix it if you like.  Let me know if you need something.
> 
> Tony
> 
> On 1/25/22 12:06 PM, Sebastian Stenzel wrote:
>> Hi all,
>> while playing around with JFR today, I stumbled upon a piece of code that causes a thousands of byte[] allocations. In fact it is responsible for 90% of the memory allocations in my application and causes GC to run without pause during decryption of large files.
>> The line in question can be found here [1]. The behaviour can be triggered with a simple AES-CTR cipher as in this minimal example [2].
>> As you can see, the comments in [1] even say that this byte[] _should_ only be allocated if there is insufficient space in the provided output buffer. However some kind of conditional statement is missing.
>> There used to be a `if (getMode() != GCM_MODE || outputCapacity < estOutSize)`, before GCM has been decoupled from CipherCore. But this line seems wrong to me either.
>> I guess this should be fixed, as I'm probably not the only one affected by those allocations. I'd be volunteering, unless the behaviour is intended.
>> Cheers!
>> Sebastian
>> [1] https://github.com/openjdk/jdk/blob/76fe03fe01a7c824e2e9263de95b8bcbb4b9d752/src/java.base/share/classes/com/sun/crypto/provider/CipherCore.java#L815-L818 <https://github.com/openjdk/jdk/blob/76fe03fe01a7c824e2e9263de95b8bcbb4b9d752/src/java.base/share/classes/com/sun/crypto/provider/CipherCore.java#L815-L818>
>> [2] https://gist.github.com/overheadhunter/f3969950c0fdbaecaa77c857b2246cc5 <https://gist.github.com/overheadhunter/f3969950c0fdbaecaa77c857b2246cc5>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/security-dev/attachments/20220126/95d36e5b/attachment.htm>