<div dir="ltr"><div>Hi Eddy,</div><div><br></div><div>Thank you for the investigation and the informative mail.</div><div><br></div><div>Just to clarify:</div><div><br></div><div>There is no functional problem in that the archives could be invalid, unreadable, or not standard-conform? The problem would be reproducibility and failing tests that you argue are too strict?</div><div><br></div><div>Reproducibility is obviously a problem for vendors doing reproducible builds. That could be helped with a switch. Other than that, I am not sure. I could imagine theoretical problems with database applications when compressed artifacts change unexpectedly given the same input, but nothing concrete comes to mind.</div><div><br></div><div>Thanks, Thomas</div><div><br></div><div><br></div></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Tue, Nov 4, 2025 at 8:49 AM Eduard Stefes <<a href="mailto:eddy@linux.ibm.com">eddy@linux.ibm.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello,<br>
<br>
# TLDR<br>
<br>
zlib (and zlib-ng) on s390x use hardware compression. This<br>
hardware compression implementation has currently some problems when<br>
used by openjdk.<br>
<br>
<br>
<br>
# current situation<br>
<br>
- openjdk uses zlib(or zlib-ng) for most(all?) its data compression.<br>
This includes also jar file creation.<br>
- s390x has deflate implemented on hardware level(called dfltcc). This<br>
implementation is up to 10x faster(empirical value) compared to the<br>
standard software algorithm.<br>
- The dfltcc implementation has some drawbacks vs the software<br>
implementation:<br>
- dfltcc will ALWAYS write to the output buffer, indifferent<br>
of the flushing parameter passed to deflate()<br>
- dfltcc does not generate reproducible output. This means<br>
that it will always generate valid deflate data streams<br>
that uncompress(inflate) to the same input. But the<br>
compressed data may look different between two calls with<br>
the same input data, because the hardware compressor depends<br>
also on hardware variables that not visible to the external<br>
api user.<br>
<br>
<br>
<br>
<br>
# the problem<br>
<br>
due to the differences of the hardware implementation, some tests in<br>
the openjdk JREG tests fail(tracked here[1][2])<br>
<br>
- java/util/zip/CloseInflaterDeflaterTest.java<br>
fails due to dfltcc's flushing behaviour. The test should check if<br>
the openjdk wrapper around the jni and native library will<br>
successfully detect writes to closed output streams. This was<br>
created because there where bugs with and race conditions in the<br>
write/close/flush.<br>
<br>
- tools/jlink/JLinkReproducibleTest.java:<br>
- tools/jar/modularJar/JarToolModuleDescriptorReproducibilityTest.java:<br>
This tests fail due to two calls to dfltcc will not generate the<br>
same compressed data for the same input data. I want to<br>
emphasize that RFC 1950 and 1951 do not guarantee the same output<br>
for two deflate/zlib data streams if feed the same input data.<br>
<br>
<br>
# proposed solutions<br>
## flushing problem<br>
<br>
IMHO the CloseInflaterDeflaterTest.java test relays on zlib behavior<br>
where zlib explicitly has the freedom to change its behavior[3][1].<br>
As a quick and dirty solution i would propose to backport the<br>
disablement[4] of this test to openjdk-17 21 and 24.<br>
<br>
I read and traced through the test, and for me it looks like the<br>
behavior of the openjdk and jni wrappers will be the same regardless of<br>
the underlying zlib. Therefor it seems ok to assume that: "If its okay<br>
on x86 it will be ok on s390x"<br>
<br>
However I think that this test might need a redesign. It relies on<br>
flushing behavior of zlib the the library explicitly stated can change.<br>
<br>
<br>
<br>
## reproducible compression<br>
<br>
I don't know how much openjdk relies on reproducible data compression.<br>
I assume it is preferable if its possible to create the same .jar twice<br>
and get the same binary data. The zlib dfltcc implementation could be<br>
controlled via an env variable to disable the hardware compression[5].<br>
Also usually dftcc will only be used for certain compression levels and<br>
this can also be changed via env variables[6].<br>
<br>
Unfortunately this env variables are evaluated at library load and can<br>
not be adjusted during runtime.<br>
<br>
Moreover ubuntu sets the default LEVEL_MASK to 0x7e[7]. This means<br>
that the compression level is also not a reliabl method in order to<br>
force zlib to use software instead of hardware compression.<br>
<br>
Also zlib-ng lacks this env variables. Here the use of dfltcc cannot be<br>
influenced at all.<br>
<br>
This all leads me to the conclusion that there has to be a decision if<br>
and how openjdk on s390x will be able to generate reproducible jars.<br>
For the time being I suppose to also disable this tests and backport<br>
the disablements down till openjdk-17.<br>
<br>
<br>
<br>
# Finally<br>
<br>
I hope i did not overwhelm you all with this email. As I don't have a<br>
<a href="http://bugs.openjdk.org" rel="noreferrer" target="_blank">bugs.openjdk.org</a> account, I thought the mailing list is the best place<br>
to discuss this mater.<br>
<br>
cheers Eddy<br>
<br>
<br>
<br>
[1] <a href="https://bugs.launchpad.net/ubuntu/+source/openjdk-21/+bug/2109016" rel="noreferrer" target="_blank">https://bugs.launchpad.net/ubuntu/+source/openjdk-21/+bug/2109016</a><br>
[2] <a href="https://bugs.openjdk.org/browse/JDK-8339216" rel="noreferrer" target="_blank">https://bugs.openjdk.org/browse/JDK-8339216</a><br>
[3] <a href="https://github.com/madler/zlib/blob/develop/zlib.h#L286-L288" rel="noreferrer" target="_blank">https://github.com/madler/zlib/blob/develop/zlib.h#L286-L288</a><br>
[4]<br>
<a href="https://github.com/openjdk/jdk/commit/537447f8816129dad9a1edd21bd30f3edf69ea60" rel="noreferrer" target="_blank">https://github.com/openjdk/jdk/commit/537447f8816129dad9a1edd21bd30f3edf69ea60</a><br>
[5]<br>
<a href="https://github.com/iii-i/zlib/blob/dfltcc/contrib/s390/dfltcc.c#L705-L706" rel="noreferrer" target="_blank">https://github.com/iii-i/zlib/blob/dfltcc/contrib/s390/dfltcc.c#L705-L706</a><br>
[6]<br>
<a href="https://github.com/iii-i/zlib/blob/dfltcc/contrib/s390/dfltcc.c#L714-L715" rel="noreferrer" target="_blank">https://github.com/iii-i/zlib/blob/dfltcc/contrib/s390/dfltcc.c#L714-L715</a><br>
[7] <a href="https://git.launchpad.net/ubuntu/+source/zlib/tree/debian/rules#n53" rel="noreferrer" target="_blank">https://git.launchpad.net/ubuntu/+source/zlib/tree/debian/rules#n53</a><br>
</blockquote></div>