AES ctr benchmark performance

[Adam Petcher]

Another thing I noticed: The AES microbenchmark tests the performance of 
Cipher.doFinal(byte[]), which allocates an array on each call. This is 
(arguably) a bug in the benchmark, so I created JDK-8214800 to track it. 
Based on my experiments, this bug accounts for around 30% of the 
difference between OpenSSL performance and the JDK performance, as 
reported by this benchmark.

On 12/3/2018 5:30 PM, Adam Petcher wrote:
> I'm seeing JDK 11 performance around 40% of OpenSSL performance, too. 
> This is on a Core i7-4980HQ (Haswell, turbo at 4 GHz).
>
> In addition to the "Java stuff" (e.g. garbage collection, array bounds 
> checking) that Tony suggested, there is also the issue that the JDK 
> native implementation does not interleave instructions, but the 
> OpenSSL implementation does. The OpenSSL source file[1] has some great 
> information about the effect of interleaving and the performance on 
> various platforms. The JDK implementation of AES-CTR encrypts 6 blocks 
> at a time, but it does all the other operations (increment counter, 
> xor, etc) in sequence with the aesenc operations. The OpenSSL 
> implementation performs all of these other operations in parallel with 
> the aesenc operations. This difference in the native implementation 
> likely accounts for some of the difference in performance, but I don't 
> know how much. The "other" operations only require a relatively small 
> number of cycles, but doing them in sequence may hurt performance a 
> lot due to pipeline stalls.
>
> [1] 
> https://github.com/openssl/openssl/blob/master/crypto/aes/asm/aesni-x86_64.pl
>
> On 12/3/2018 1:45 PM, Anthony Scarpino wrote:
>> Hi,
>>
>> That is how I read it, but when I've done the test it's closer to 
>> 55%. But openssl speed and jmh are not comparable.  Openssl is only 
>> running through it's algorithm, while jmh is running through the VM 
>> and java byte code. jmh has to do more work outside of pure crypto 
>> ops than openssl.
>>
>> Tony
>>
>>
>> On 12/3/18 12:09 AM, Kasper Janssens wrote:
>>> Hello,
>>>
>>> I've been measuring the performance of the java 11 built in AES-NI 
>>> instructions and, with NI switched on through JAVA_TOOL_OPTIONS = 
>>> ‘-XX:+UnlockDiagnosticVMOptions -XX:+UseAES -XX:+UseAESIntrinsics’.
>>>
>>> I run the benchmarks that are present in the open jdk itself 
>>> through  ‘java -server -jar 
>>> target/jmh-jdk-microbenchmarks-1.0-SNAPSHOT.jar 
>>> org.openjdk.bench.javax.crypto.full.AESBench.encrypt -p 
>>> algorithm=AES/CTR/NoPadding’.
>>>
>>> When I compare the results and calculate the throughput in GB/s, for 
>>> 16k encoding blocks, I get about 40% of the throughput when compared 
>>> to the openssl performance test through ‘openssl speed -evp 
>>> aes-128-ctr’.
>>>
>>> I don't seem to find a decent comparison/benchmark that describes 
>>> which percentage of the maximum performance to expect with jdk 11, 
>>> but 40 % seems rather low. Does anybody know of a comparison (or 
>>> might have an idea where I go wrong,  because, as these are off the 
>>> shelf perf tools I would think not a lot can be tweaked there).
>>>
>>> For reference, the openssl test reports this:
>>>
>>> The 'numbers' are in 1000s of bytes per second processed.
>>>
>>> type             16 bytes     64 bytes    256 bytes   1024 bytes   
>>> 8192 bytes  16384 bytes
>>>
>>> aes-128-ctr     583576.76k  1907072.21k  4144141.40k 5612174.68k  
>>> 6252751.53k  6258371.24k
>>>
>>> So, for 16k blocks, 6258371k bytes per second
>>>
>>> The openjdk benchmark reports this
>>>
>>> Benchmark               (algorithm)  (dataSize)  (keyLength) 
>>> (provider)   Mode  Cnt       Score     Error  Units
>>>
>>> AESBench.encrypt  AES/CTR/NoPadding       16384 128              
>>> thrpt  100  168066.873 ± 488.873  ops/s
>>>
>>> Out of which I conclude that, for 16k blocks, the perf here is 
>>> 168066 * 16k or about 2689056k bytes per second or about 40 % of the 
>>> openssl number, taking some liberty with rounding the numbers.
>>>
>>> Is that expected or am I drawing the wrong conclusions? Or am I 
>>> missing something?
>>>
>>> When I print the options, the jvm reports that the AES intrinsics 
>>> are being used :
>>>
>>> java -XX:+PrintFlagsFinal | grep -i AES
>>>
>>>       intx MaxBCEAEstimateLevel                     = 
>>> 5                                        {product} {default}
>>>
>>>       intx MaxBCEAEstimateSize                      = 
>>> 150                                      {product} {default}
>>>
>>>       bool UseAES                                   = 
>>> true                                     {product} {default}
>>>
>>> System specs : Ubuntu 18.04
>>>
>>> openjdk version "10.0.2" 2018-07-17
>>>
>>> OpenJDK Runtime Environment (build 10.0.2+13-Ubuntu-1ubuntu0.18.04.4)
>>>
>>> OpenJDK 64-Bit Server VM (build 10.0.2+13-Ubuntu-1ubuntu0.18.04.4, 
>>> mixed mode)
>>>
>>> Kasper
>>>
>>