RFR: 8299748: java/util/zip/Deinflate.java failing on s390x [v2]

[Jaikiran Pai]

On Thu, 2 Feb 2023 08:27:55 GMT, Amit Kumar <amitkumar at openjdk.org> wrote:

>> DeInflate.java test fails on s390x platform because size for out1 array which is responsible for storing the compressed data is insufficient. And being unable to write whole compressed data on array, on s390 whole data can't be recovered after compression. So this fix increase Array size (for s390).
>
> Amit Kumar has updated the pull request incrementally with one additional commit since the last revision:
> 
>   change acc to Alan comments

Hello Amit, like Alan notes, we don't have a clarity on what's going on here. It is an interesting failure, but neither of us have access to the system where this fails, so we can just provide suggestions on how to narrow this down. Alan provided some suggestions on what can be done to debug this. I think you or someone who has access to this system will need to do some more debugging/changes to narrow this down.

Your logs so far have shown that this fails for deflate level 1. What you can do in the test is just focus on this specific level and the default strategy and then debug the code (either through logs or actual runtime debugging) to figure out why "deflate.finished()" returns false. To help, I have trimmed down the test case to this specific scenario and here's the patch:


diff --git a/test/jdk/java/util/zip/DeInflate.java b/test/jdk/java/util/zip/DeInflate.java
index ff2cb63c6e7..d720c53e8c3 100644
--- a/test/jdk/java/util/zip/DeInflate.java
+++ b/test/jdk/java/util/zip/DeInflate.java
@@ -125,9 +125,30 @@ public class DeInflate {
         Arrays.fill(out1, (byte)0);
         Arrays.fill(out2, (byte)0);
 
+        System.out.println("Feeding input of length " + len);
         def.setInput(in, 0, len);
-        def.finish();
-        int m = def.deflate(out1);
+        def.finish(); // this implies the deflater will not be provided with any more input data
+                      // than what we set on previous line
+
+        int m = 0;
+        while (!def.finished()) {
+            int remaining = out1.length - m;
+            int numWritten = def.deflate(out1, m, remaining);
+            System.out.println("Num written " + numWritten);
+            if (numWritten == 0) {
+                // Check if needsInput(), this shouldn't have happened because we have
+                // marked the deflater with "def.finish()" indicating there's no more input data to
+                // provide
+                if (def.needsInput()) {
+                    throw new RuntimeException("Deflater is expecting more input data when " +
+                            "deflater.finish() was already called");
+                } else {
+                    System.out.println("Deflater isn't expecting any more data but numWritten is 0" +
+                            " and finished is " + def.finished());
+                }
+            }
+            m += numWritten;
+        }
 
         Inflater inf = new Inflater(nowrap);
         inf.setInput(out1, 0, m);
@@ -271,34 +292,10 @@ public class DeInflate {
         byte[] dataOut1 = new byte[dataIn.length + 1024];
         byte[] dataOut2 = new byte[dataIn.length];
 
-        Deflater defNotWrap = new Deflater(Deflater.DEFAULT_COMPRESSION, false);
-        Deflater defWrap = new Deflater(Deflater.DEFAULT_COMPRESSION, true);
-
-        for (int level = Deflater.DEFAULT_COMPRESSION;
-                 level <= Deflater.BEST_COMPRESSION; level++) {
-            for (int strategy = Deflater.DEFAULT_STRATEGY;
-                     strategy <= Deflater.HUFFMAN_ONLY; strategy++) {
-                for (boolean dowrap : new boolean[] { false, true }) {
-                    System.out.println("level:" + level +
-                                     ", strategy: " + strategy +
-                                     ", dowrap: " + dowrap);
-                    for (int i = 0; i < 5; i++) {
-                        int len = (i == 0)? dataIn.length
-                                          : new Random().nextInt(dataIn.length);
-                        // use a new deflater
-                        Deflater def = newDeflater(level, strategy, dowrap, dataOut2);
-                        check(def, dataIn, len, dataOut1, dataOut2, dowrap);
-                        def.end();
-
-                        // reuse the deflater (with reset) and test on stream, which
-                        // uses a "smaller" buffer (smaller than the overall data)
-                        def = resetDeflater(dowrap ? defWrap : defNotWrap, level, strategy);
-                        checkStream(def, dataIn, len, dataOut1, dataOut2, dowrap);
-                    }
-                }
-                // test setDictionary()
-                checkDict(level, strategy);
-            }
-        }
+        final boolean dowrap = true;
+        Deflater def = newDeflater(Deflater.BEST_SPEED, Deflater.DEFAULT_STRATEGY, dowrap, dataOut2);
+        check(def, dataIn, dataIn.length, dataOut1, dataOut2, dowrap);
+        def.end();
+
     }
 }

(apply it afresh, without any existing changes).
What this patch does is, it jus focuses on the level 1 deflater. Additionally in the check() method is adds some additional checks to see why the deflater might be returning finished() false. The (original and this patched version of the) test calls `deflater.finish()` before calling `deflate()`. What this means is that the deflater shouldn't expect any more input data. I have added some checks in that test to see if it does infact expect more input data. 

After running the test you can share with us the logs. We could provide more suggestions what to debug further, but I think you or someone with access to that system will have to do the actual investigation.

-------------

PR: https://git.openjdk.org/jdk/pull/12283