RFR[M]: Adding MD5 Intrinsic on x86-64

Ludovic Henry luhenry at microsoft.com
Tue Aug 4 04:07:49 UTC 2020


Updated webrev: http://cr.openjdk.java.net/~luhenry/8250902/webrev.02

> Next code in inline_digestBase_implCompressMB should be reversed (get_long_*() should be called for long_state):
>
>    if (long_state) {
>      state = get_state_from_digestBase_object(digestBase_obj);
>    } else {
>      state = get_long_state_from_digestBase_object(digestBase_obj);
>    }

Thanks for pointing that out. I tested everything with `hotspot:tier1` and `jdk:tier1` in fastdebug on Windows-x86, Windows-x64 and Linux-x64.

> It seems that the algorithm can be optimized further using SSE/AVX instructions. I am not aware of any specific SSE/AVX implementation which leverages those instructions in the best possible way. Sandhya can chime in more on that.

I have done some research prior to implementing this intrinsic and the only pointers I could find to vectorized MD5 is on computing _multiple_ MD5 hashes in parallel but not a _single_ MD5 hash. Using vectors effectively parallelize the computation of many MD5 hash, but it does not accelerate the computation of a single MD5 hash. And looking at the algorithm, every step depends on the previous step's result, which make it particularly hard to parallelize/vectorize.

> As far as I know, I came across this which points to MD5 SSE/AVX implementation. https://software.intel.com/content/www/us/en/develop/articles/intel-isa-l-cryptographic-hashes-for-cloud-storage.html

That library points to computing many MD5 hashes in parallel. Quoting: "Intel® ISA-L uses a novel technique called multi-buffer hashing, which [...] compute several hashes at once within a single core." That is similar to what I found in researching how to vectorize MD5. I also did not find any reference of an ISA-level implementation of MD5, neither in x86 nor ARM.

If you can point me to a document describing how to vectorize MD5, I would be more than happy to take a look and implement the algorithm. However, my understanding is that MD5 is not vectorizable by-design.

> Add tests to verify intrinsic implementation. You can use test/hotspot/jtreg/compiler/intrinsics/sha/ as examples.

I looked at these tests and they already cover MD5. I am not sure what's the best way to add tests here: 1. should I rename ` compiler/intrinsics/sha` to ` compiler/intrinsics/digest` and add the md5 tests there, 2. should I just add ` compiler/intrinsics/md5`, or 3. the name doesn't matter and I can just add it in ` compiler/intrinsics/sha`?

> In vm_version_x86.cpp move UseMD5Intrinsics flag setting near UseSHA flag setting.

Fixed.

> In new file macroAssembler_x86_md5.cpp no need empty line after copyright line. There is also typo 'rrdistribute':
>
>   * This code is free software; you can rrdistribute it and/or modify it
>
> Our validate-headers check failed. See GPL header template: ./make/templates/gpl-header

I updated the header, and added the license for the original code for the MD5 core algorithm.

> Did you test it on 32-bit x86?

I did run `hotspot:tier1` and `jdk:tier1` on Windows-x86, Windows-x64 and Linux-x64.

> Would be interesting to see result of artificially switching off AVX and SSE:
> '-XX:UseSSE=0 -XX:UseAVX=0'. It will make sure that only general instructions are needed.

The results are below:

-XX:-UseMD5Intrinsics
Benchmark              (digesterName)  (length)  (provider)   Mode  Cnt     Score    Error   Units
MessageDigests.digest             md5        64     DEFAULT  thrpt   10  3512.618 ± 9.384  ops/ms
MessageDigests.digest             md5      1024     DEFAULT  thrpt   10   450.037 ± 1.213  ops/ms
MessageDigests.digest             md5     16384     DEFAULT  thrpt   10    29.887 ± 0.057  ops/ms
MessageDigests.digest             md5   1048576     DEFAULT  thrpt   10     0.485 ± 0.002  ops/ms

-XX:+UseMD5Intrinsics
Benchmark              (digesterName)  (length)  (provider)   Mode  Cnt     Score   Error   Units
MessageDigests.digest             md5        64     DEFAULT  thrpt   10  4212.156 ± 7.781  ops/ ms => 19% speedup
MessageDigests.digest             md5      1024     DEFAULT  thrpt   10   548.609 ± 1.374  ops/ ms => 22% speedup
MessageDigests.digest             md5     16384     DEFAULT  thrpt   10    37.961 ± 0.079  ops/ ms => 27% speedup
MessageDigests.digest             md5   1048576     DEFAULT  thrpt   10     0.596 ± 0.006  ops/ ms => 23% speedup

-XX:-UseMD5Intrinsics -XX:UseSSE=0 -XX:UseAVX=0
Benchmark              (digesterName)  (length)  (provider)   Mode  Cnt     Score    Error   Units
MessageDigests.digest             md5        64     DEFAULT  thrpt   10  3462.769 ± 4.992  ops/ms
MessageDigests.digest             md5      1024     DEFAULT  thrpt   10   443.858 ± 0.576  ops/ms
MessageDigests.digest             md5     16384     DEFAULT  thrpt   10    29.723 ± 0.480  ops/ms
MessageDigests.digest             md5   1048576     DEFAULT  thrpt   10     0.470 ± 0.001  ops/ms

-XX:+UseMD5Intrinsics -XX:UseSSE=0 -XX:UseAVX=0
Benchmark              (digesterName)  (length)  (provider)   Mode  Cnt     Score   Error   Units
MessageDigests.digest             md5        64     DEFAULT  thrpt   10  4237.219 ± 15.627  ops/ms => 22% speedup
MessageDigests.digest             md5      1024     DEFAULT  thrpt   10   564.625 ±  1.510  ops/ms => 27% speedup
MessageDigests.digest             md5     16384     DEFAULT  thrpt   10    38.004 ±  0.078  ops/ms => 28% speedup
MessageDigests.digest             md5   1048576     DEFAULT  thrpt   10     0.597 ±  0.002  ops/ms => 27% speedup

Thank you,
Ludovic


More information about the hotspot-compiler-dev mailing list