javac doesn't work with jar files with >64k entries

[Martin Buchholz]

On Wed, Nov 7, 2012 at 10:20 AM, Kumar Srinivasan <
kumar.x.srinivasan at oracle.com> wrote:

>  Hi Martin!,
>
> I believe I fixed this in jdk7 the changesets that you might find
> interesting are:
> http://hg.openjdk.java.net/jdk7/jdk7/langtools/rev/6970d9fb8e02
> http://hg.openjdk.java.net/jdk7/jdk7/langtools/rev/3d45cc94ee0f
>
> We use javac's fast zip implementation first if it fails then we use the
> platforms Zip implementation, by using a flag you can control which one to
> use.
>
>
I tried to reproduce the bug with latest jdk7 (7u10-ea) and jdk8 (b62) and
it appears to still be there as well.

Fallback may indeed be working for ZIP64 input jars (I haven't tried it)
but it does *not* work for zip files with >64k entries, because javac's zip
implementation believes it has correctly read all the entries, when in fact
it has only read ENDTOT entries.

I haven't tried to create a minimal test (we have a non-minimal one), but
it should be easy to create an input zip with 64k bogus empty file entries
followed by some real class files at the end.

Martin

Hope this helps.
>
> Kumar
>
>
>
>  Hi compiler folk,
>
>  You don't like someone else's zip implementation, cuz it's too SLOOOW,
> so you write your own.  Now you have two problems.
>
>  ./share/classes/com/sun/tools/javac/file/ZipFileIndex.java:571:
>    int entryCount = get2ByteLittleEndian(zipDir, 0);
>
>  The ZIP specification includes ENDTOT, a field that contains the number
> of entries in a zip file.
> Unfortunately, all high quality zip implementations ignore this field (or
> more accurately, treat it as a hint).
> Read zip_util.c and be enlightened:
>
>      /* Initialize zip file data structures based on the total number
>      * of central directory entries as stored in ENDTOT.  Since this
>      * is a 2-byte field, but we (and other zip implementations)
>      * support approx. 2**31 entries, we do not trust ENDTOT, but
>      * treat it only as a strong hint.  When we call ourselves
>
>  This needs fixing.  Since you are iterating through the CEN already,
> just stop when you hit the end header, whose location you know.
>
>  You can imagine the painful frustrating failure mode of javac starting
> to very mysteriously fail as the size of an input jar creeps over 64k
> entries.
>
>  jdk6 javac has the same problem, but disabling the "optimized" zip
> implementation requires different mechanisms.
>
>  The sources could be more readable.  Probably your source code is
> haunted by Phil Katz' ghost because you are using hex constants instead of
> 'P' and 'K' as Phil Katz intended.
>
>                          !(endbuf[i] == 0x50 &&
>                         endbuf[i + 1] == 0x4b &&
>
>  You get points for noticing that the input zip file is in ZIP64 format
>
>                      if (sz < 0 || get2ByteLittleEndian(zipDir, 0) ==
> 0xffff) {
>                         throw new ZipFormatException("detected a zip64
> archive");
>                     }
>
>  but the only proper reaction is to either add full ZIP64 support or
> fallback to the "unoptimized but actually working" implementation.
>
>  Martin
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/compiler-dev/attachments/20121107/c398a913/attachment.html