javac doesn't work with jar files with >64k entries

[Kumar Srinivasan]

On 11/7/2012 10:59 AM, Martin Buchholz wrote:
>
>
> On Wed, Nov 7, 2012 at 10:20 AM, Kumar Srinivasan 
> <kumar.x.srinivasan at oracle.com <mailto:kumar.x.srinivasan at oracle.com>> 
> wrote:
>
>     Hi Martin!,
>
>     I believe I fixed this in jdk7 the changesets that you might find
>     interesting are:
>     http://hg.openjdk.java.net/jdk7/jdk7/langtools/rev/6970d9fb8e02
>     http://hg.openjdk.java.net/jdk7/jdk7/langtools/rev/3d45cc94ee0f
>
>     We use javac's fast zip implementation first if it fails then we
>     use the
>     platforms Zip implementation, by using a flag you can control
>     which one to use.
>
>
> I tried to reproduce the bug with latest jdk7 (7u10-ea) and jdk8 (b62) 
> and it appears to still be there as well.
>
> Fallback may indeed be working for ZIP64 input jars (I haven't tried 
> it) but it does *not* work for zip files with >64k entries, because 
> javac's zip implementation believes it has correctly read all the 
> entries, when in fact it has only read ENDTOT entries.

Talking about tests, take a look at 
langtools/test/tools/javac/file/zip/T6836682.java
Scenario A, that does not cut it ?

Kumar

>
> I haven't tried to create a minimal test (we have a non-minimal one), 
> but it should be easy to create an input zip with 64k bogus empty file 
> entries followed by some real class files at the end.
> Martin
>
>     Hope this helps.
>
>     Kumar
>
>
>
>>     Hi compiler folk,
>>
>>     You don't like someone else's zip implementation, cuz it's too
>>     SLOOOW, so you write your own.  Now you have two problems.
>>
>>     ./share/classes/com/sun/tools/javac/file/ZipFileIndex.java:571:  
>>              int entryCount = get2ByteLittleEndian(zipDir, 0);
>>
>>     The ZIP specification includes ENDTOT, a field that contains the
>>     number of entries in a zip file.
>>     Unfortunately, all high quality zip implementations ignore this
>>     field (or more accurately, treat it as a hint).
>>     Read zip_util.c and be enlightened:
>>
>>         /* Initialize zip file data structures based on the total number
>>          * of central directory entries as stored in ENDTOT.  Since this
>>          * is a 2-byte field, but we (and other zip implementations)
>>          * support approx. 2**31 entries, we do not trust ENDTOT, but
>>          * treat it only as a strong hint.  When we call ourselves
>>
>>     This needs fixing.  Since you are iterating through the CEN
>>     already, just stop when you hit the end header, whose location
>>     you know.
>>
>>     You can imagine the painful frustrating failure mode of javac
>>     starting to very mysteriously fail as the size of an input jar
>>     creeps over 64k entries.
>>
>>     jdk6 javac has the same problem, but disabling the "optimized"
>>     zip implementation requires different mechanisms.
>>
>>     The sources could be more readable.  Probably your source code is
>>     haunted by Phil Katz' ghost because you are using hex constants
>>     instead of 'P' and 'K' as Phil Katz intended.
>>
>>                             !(endbuf[i] == 0x50 &&
>>                             endbuf[i + 1] == 0x4b &&
>>
>>     You get points for noticing that the input zip file is in ZIP64
>>     format
>>
>>                         if (sz < 0 || get2ByteLittleEndian(zipDir, 0)
>>     == 0xffff) {
>>                             throw new ZipFormatException("detected a
>>     zip64 archive");
>>                         }
>>
>>     but the only proper reaction is to either add full ZIP64 support
>>     or fallback to the "unoptimized but actually working" implementation.
>>
>>     Martin
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/compiler-dev/attachments/20121107/59cf73ae/attachment.html