com.sun.crypto.provider.GHASH performance fix

[Tim Whittington]

On 16/01/2015, at 9:18 pm, Anthony Scarpino <anthony.scarpino at oracle.com> wrote:

>> On Jan 15, 2015, at 12:26 PM, Florian Weimer <fweimer at redhat.com> wrote:
>> 
>> Yes, they are going to help quite a bit as well.  The other thing we need to fix for TLS is that AES-GCM is a garbage collector stress test.  Last time I looked, for each transferred byte, there were four bytes allocated on the heap.
> 
> What exactly are you referring to here?  Is there a problem in TLS where it is using too much memory for AES-GCM specifically or AES-GCM itself is using too much memory?  What test did you do to see this heap allocation or is this code inspection?
> 

The GCM implementation (jdk/src/share/classes/com/sun/crypto/provider/GaloisCounterMode.java) does a lot of unnecessary data copying/allocation:

- AAD is buffered in a ByteArrayOutputStream and accessed with toByteArray() = unnecessary copy of all AAD to buffer and to new array
- on decrypt, all ciphertext is buffered in a ByteArrayOutputStream, accessed with toByteArray() = unnecessary copying of all ciphertext data to new array (twice if you don’t buy that it should buffer ciphertext)
- all of the block pad/length block operations allocate new arrays (they could all use a preallocated per-cipher buffer)
- there are a bunch of other byte[] allocations that could probably be eliminated

ByteArrayOutputStream buffer can be trivially accessed to avoid copy, and the AAD buffer can probably be avoided entirely.

I haven’t checked to see if JSSE is using the in[], out[] variants of update/doFinal to avoid copying there.

cheers
tim

> thanks
> 
> Tony
>