RFR: 8156484: ZipFile retains too much native memory from caching Inflaters

Peter Levart peter.levart at gmail.com
Mon May 23 21:28:35 UTC 2016


Hi Sherman, Martin,


On 05/23/2016 08:28 PM, Xueming Shen wrote:
> My apology. I meant to say "to have Inflater to be aware of the 
> ObjectPool...".
> The proposed change switches from the finalizer to the ObjectPool to 
> clean up
> the native resource for Inflater. It appears to be a bigger change. 
> Which has
> a default 32/maxCacheSize and 1 secod keepAliveTime setting. It might have
> a big impact to existing applications on how to use the Inflater. It 
> would be
> fine to have the ZipFile to arrange how to use the Inflater based on 
> its use
> scenario, but I'm concerned if it's the correct approach to change the 
> default
> behavior of Inflater, which might have nothing to do with ZipFile.
>
> sherman

It might have a big impact to existing applications, yes. A positive one!



On 05/23/2016 09:21 PM, Martin Buchholz wrote:
> ....
> I am still contending that zstream caching is not worth much.  The
> best I could do with a synthetic benchmark designed to make caching
> win was 25% better performance.  Martin's rule is if you can't even
> come up with a friendly benchmark that gives you a 2x win, give up on
> your optimization!  So performance wise, there is no big impact here
> on any applications, except to fix excessive memory retention and
> OOME.
>
>


Here's the 1st test I did. The micro benchmark Martin created and is 
attached to the issue 8156484. Just modified a bit to issue a heap dump 
at the end:

public class Benchmark60kMulti {

     private static void dumpHistogram() {
         // dump a heap histogram
         try {
             new ProcessBuilder(
                 System.getProperty("java.home") + "/bin/jmap",
                 "-histo:live", 
String.valueOf(ProcessHandle.current().getPid())
             ).redirectOutput(ProcessBuilder.Redirect.to(
                 new File("histo_live.txt"))
             ).start().waitFor();
         } catch (Exception e) {
             e.printStackTrace();
         }
     }

     public static void main(String[] args) throws Throwable {
         final int REPS = 20;
         final int THREADS = 8;
         final byte[] data = "123456789".getBytes("UTF-8");
         final File file = new File("60k.zip");

         if (!file.exists()) {
             try (FileOutputStream fos = new FileOutputStream(file);
                  BufferedOutputStream bos = new BufferedOutputStream(fos);
                  ZipOutputStream zos = new ZipOutputStream(bos)) {
                 for (int i = 0; i < 60_000; i++) {
                     ZipEntry ze = new ZipEntry(i + ".txt");
                     ze.setMethod(ZipEntry.DEFLATED);
                     ze.setSize(0);
                     ze.setCrc(0);
                     zos.putNextEntry(ze);
                     zos.write(data);
                 }
             }
         }

         Path target = Paths.get("60k.zip");
         for (int i = 0; i < THREADS; i++) {
             Path link = Paths.get("symlink-" + i + ".zip");
             if (!link.toFile().exists())
                 Files.createSymbolicLink(link, target);
         }

         ExecutorService pool = Executors.newFixedThreadPool(THREADS);
         ArrayList<Future<?>> futures = new ArrayList<>();

         for (int i = 0; i < THREADS; i++) {
             final Path link = Paths.get("symlink-" + i + ".zip");
             final Runnable task = () -> {
                 try (ZipFile zipFile = new ZipFile(link.toFile())) {
                     for (int j = 0; j < REPS; j++) {
                         byte[] buf = new byte[100];
                         Enumeration<? extends ZipEntry> entries = 
zipFile.entries();
                         while (entries.hasMoreElements()) {
                             ZipEntry zentry = entries.nextElement();
                             try (InputStream is = 
zipFile.getInputStream(zentry)) {
                                 is.read(buf);
                             }
                         }
                     }
                 } catch (Throwable t) { throw new Error(t); }
             };
             futures.add(pool.submit(task));
         }
         pool.shutdown();
         for (Future future : futures) future.get();

         dumpHistogram();

         pool.awaitTermination(1, TimeUnit.DAYS);
     }
}



"time" Linux command and a top-10 heap dump of live objects:

real    0m22.197s
user    1m34.968s
sys     0m12.577s

  num     #instances         #bytes  class name (module)
-------------------------------------------------------
    1:       3621046      704046712  [B (java.base at 9-internal)
    2:       2410133       96405320  java.lang.ref.Finalizer 
(java.base at 9-internal)
    3:       1572755       75492240 
java.util.zip.ZipFile$ZipFileInputStream (java.base at 9-internal)
    4:       1204735       67465160 
java.util.zip.ZipFile$ZipFileInflaterInputStream (java.base at 9-internal)
    5:          4597         147104  java.util.HashMap$Node 
(java.base at 9-internal)
    6:          5875         141000  java.lang.String (java.base at 9-internal)
    7:          1140         138656  java.lang.Class (java.base at 9-internal)
    8:          2681          85792 
java.util.concurrent.ConcurrentHashMap$Node (java.base at 9-internal)
    9:          1938          84760  [Ljava.lang.Object; 
(java.base at 9-internal)
   10:           636          61056 [Ljava.util.HashMap$Node; 
(java.base at 9-internal)


With webrev.03 applied:

real    0m6.734s
user    0m32.937s
sys     0m13.760s

  num     #instances         #bytes  class name (module)
-------------------------------------------------------
    1:          6903         483592  [B (java.base at 9-internal)
    2:          4598         147136  java.util.HashMap$Node 
(java.base at 9-internal)
    3:          1177         143112  java.lang.Class (java.base at 9-internal)
    4:          5936         142464  java.lang.String (java.base at 9-internal)
    5:          2681          85792 
java.util.concurrent.ConcurrentHashMap$Node (java.base at 9-internal)
    6:          1943          84608  [Ljava.lang.Object; 
(java.base at 9-internal)
    7:             2          65568 [Ljava.util.concurrent.ForkJoinTask; 
(java.base at 9-internal)
    8:           637          61136 [Ljava.util.HashMap$Node; 
(java.base at 9-internal)
    9:            11          33696  [C (java.base at 9-internal)
   10:           637          30576  java.util.HashMap 
(java.base at 9-internal)



I think this is more than 2x win in speed, don't you? Not to mention memory.

Regards, Peter




More information about the core-libs-dev mailing list