ZipFileSystem and the impact of JDK-8031748 on ordering of MANIFEST files in the created jars

[Lance Andersen]

Hi Jaikarian,

This issue has been looked at several times over the years (see https://bugs.openjdk.java.net/browse/JDK-5046178 <https://bugs.openjdk.java.net/browse/JDK-5046178> as an example) WRT JarInpuStream/MANIFEST

There is an existing Zip FS enhancement request, https://bugs.openjdk.java.net/browse/JDK-8211917.  

Best
Lance

> On Nov 14, 2019, at 10:40 AM, Jaikiran Pai <jai.forums2013 at gmail.com> wrote:
> 
> Adding core-libs-dev, since this also involves java.util.jar APIs.
> 
> -Jaikiran
> 
> On 14/11/19 8:47 PM, Jaikiran Pai wrote:
>> Please consider the code listed at the end of this mail. What it does is
>> uses ZipFileSystem to create 2 jar files with identical content. foo.jar
>> and bar.jar. Both these jar files contain a (random) text file and a
>> META-INF/MANIFEST.MF.
>> 
>> In case of foo.jar, the text file gets created first and then the
>> META-INF/MANIFEST.MF and the filesystem finally gets closed
>> 
>> In case of bar.jar, the META-INF/MANIFEST.MF gets created first and then
>> the text file and the filesystem finally gets closed.
>> 
>> Once both these (identical) jars are created, the JarInputStream class
>> is then used to open these jars and get hold of the Manifest file. What
>> results is - the JarInputStream returns a null Manifest for foo.jar (the
>> one where the META-INF/MANIFEST.MF wasn't created first), whereas the
>> JarInputStream for bar.jar rightly finds the Manifest and its correct
>> content.
>> 
>> First of all it's a surprise that the JarInputStream seemingly "loses"
>> the Manifest if the META-INF/MANIFEST.MF wasn't created first. But given
>> that it's already a known issue reported in JBS
>> https://bugs.openjdk.java.net/browse/JDK-8031748, at least we (the
>> libraries that create jar files know what to do or how to deal with it).
>> 
>> However, when using something like a zipfs FileSystem, where the code
>> which deals with writing out content to the filesystem using
>> standard/basic java.nio.file.Path and outputstreams, its hard to keep
>> track (within the libraries or user code) of the order in which the
>> files get written out. So is there any way this (undocumented)
>> requirement be implemented as an internal detail within the zipfs
>> filesystem implementation, such that it orders the META-INF/MANIFEST.MF
>> entry correctly? Or is there a way JarInputStream itself can be fixed to
>> not mandate this requirement?
>> 
>> This issue was reported in the Quarkus project
>> https://github.com/quarkusio/quarkus/issues/5399 and a workaround has
>> been proposed, but given how involved the code is (unrelated to this
>> issue), it's going to become more and more difficult to manage this
>> ordering.
>> 
>> 
>> Code reproducing this issue follows:
>> 
>> import java.io.*;
>> import java.nio.file.*;
>> import java.util.*;
>> import java.util.jar.*;
>> import java.net.*;
>> import static java.nio.file.StandardOpenOption.*;
>> 
>> public class ZipFSJar {
>>     public static void main(final String[] args) throws Exception {
>>         final Map<String, String> options =
>> Collections.singletonMap("create", "true");
>>         final Path jarPath = Paths.get("foo.jar");
>>         try (final FileSystem zipFs =
>> FileSystems.newFileSystem(URI.create("jar:" + jarPath.toUri()), options)) {
>>             // first write non-manifest content
>>             writeTxtFile(zipFs.getPath("foo.txt"));
>>             // now write manifest
>>             Files.createDirectories(zipFs.getPath("META-INF"));
>>             final Path manifestPath = zipFs.getPath("META-INF",
>> "MANIFEST.MF");
>>             writeManifest(manifestPath, "foo.bar.Baz");
>>         }
>> 
>>         // repeat for bar.jar but with manifest being written out first
>>         final Path barJar = Paths.get("bar.jar");
>>         try (final FileSystem zipFs =
>> FileSystems.newFileSystem(URI.create("jar:" + barJar.toUri()), options)) {
>>             Files.createDirectories(zipFs.getPath("META-INF"));
>>             final Path manifestPath = zipFs.getPath("META-INF",
>> "MANIFEST.MF");
>>             // first write manifest content
>>             writeManifest(manifestPath, "foo.bar.Baz");
>>             // now write text file
>>             writeTxtFile(zipFs.getPath("foo.txt"));
>>         }
>> 
>>         // now check the jar contents
>>         final Path[] jars = new Path[] {jarPath, barJar};
>>         for (final Path p : jars) {
>>             try (InputStream fileInputStream = new
>> FileInputStream(p.toFile());
>>                 final JarInputStream jaris = new
>> JarInputStream(fileInputStream);) {
>>                 final Manifest manifest = jaris.getManifest();
>>                 if (manifest == null) {
>>                     System.err.println(p + " is missing the manifest file");
>>                 } else {
>>                     System.out.println(p + " has the manifest file");
>>                     final String mainClass =
>> manifest.getMainAttributes().getValue(Attributes.Name.MAIN_CLASS);
>>                     if (!"foo.bar.Baz".equals(mainClass)) {
>>                         System.err.println("Found unexpected main class
>> " + mainClass + " in jar " + p);
>>                     }
>>                 }
>>             }
>>         }
>> 
>>     }
>> 
>>     private static void writeTxtFile(final Path filePath) throws
>> IOException {
>>         try (final OutputStream os = Files.newOutputStream(filePath)) {
>>             final byte[] someData = new byte[]{'b', 'c', 'd'};
>>             os.write(someData);
>>         }
>>     }
>> 
>>     private static void writeManifest(final Path manifestPath, final
>> String mainClass) throws IOException {
>>         try (final OutputStream os = Files.newOutputStream(manifestPath)) {
>>             final Manifest manifest = new Manifest();
>>             final Attributes attributes = manifest.getMainAttributes();
>>             attributes.put(Attributes.Name.MANIFEST_VERSION, "1.0");
>>             attributes.put(Attributes.Name.MAIN_CLASS, mainClass);
>>             manifest.write(os);
>>         }
>>     }
>> }
>> 

 <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
 <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive 
Burlington, MA 01803
Lance.Andersen at oracle.com <mailto:Lance.Andersen at oracle.com>