ZipFileSystem and the impact of JDK-8031748 on ordering of MANIFEST files in the created jars

[Jaikiran Pai]

Adding core-libs-dev, since this also involves java.util.jar APIs.

-Jaikiran

On 14/11/19 8:47 PM, Jaikiran Pai wrote:
> Please consider the code listed at the end of this mail. What it does is
> uses ZipFileSystem to create 2 jar files with identical content. foo.jar
> and bar.jar. Both these jar files contain a (random) text file and a
> META-INF/MANIFEST.MF.
>
> In case of foo.jar, the text file gets created first and then the
> META-INF/MANIFEST.MF and the filesystem finally gets closed
>
> In case of bar.jar, the META-INF/MANIFEST.MF gets created first and then
> the text file and the filesystem finally gets closed.
>
> Once both these (identical) jars are created, the JarInputStream class
> is then used to open these jars and get hold of the Manifest file. What
> results is - the JarInputStream returns a null Manifest for foo.jar (the
> one where the META-INF/MANIFEST.MF wasn't created first), whereas the
> JarInputStream for bar.jar rightly finds the Manifest and its correct
> content.
>
> First of all it's a surprise that the JarInputStream seemingly "loses"
> the Manifest if the META-INF/MANIFEST.MF wasn't created first. But given
> that it's already a known issue reported in JBS
> https://bugs.openjdk.java.net/browse/JDK-8031748, at least we (the
> libraries that create jar files know what to do or how to deal with it).
>
> However, when using something like a zipfs FileSystem, where the code
> which deals with writing out content to the filesystem using
> standard/basic java.nio.file.Path and outputstreams, its hard to keep
> track (within the libraries or user code) of the order in which the
> files get written out. So is there any way this (undocumented)
> requirement be implemented as an internal detail within the zipfs
> filesystem implementation, such that it orders the META-INF/MANIFEST.MF
> entry correctly? Or is there a way JarInputStream itself can be fixed to
> not mandate this requirement?
>
> This issue was reported in the Quarkus project
> https://github.com/quarkusio/quarkus/issues/5399 and a workaround has
> been proposed, but given how involved the code is (unrelated to this
> issue), it's going to become more and more difficult to manage this
> ordering.
>
>
> Code reproducing this issue follows:
>
> import java.io.*;
> import java.nio.file.*;
> import java.util.*;
> import java.util.jar.*;
> import java.net.*;
> import static java.nio.file.StandardOpenOption.*;
>
> public class ZipFSJar {
>     public static void main(final String[] args) throws Exception {
>         final Map<String, String> options =
> Collections.singletonMap("create", "true");
>         final Path jarPath = Paths.get("foo.jar");
>         try (final FileSystem zipFs =
> FileSystems.newFileSystem(URI.create("jar:" + jarPath.toUri()), options)) {
>             // first write non-manifest content
>             writeTxtFile(zipFs.getPath("foo.txt"));
>             // now write manifest
>             Files.createDirectories(zipFs.getPath("META-INF"));
>             final Path manifestPath = zipFs.getPath("META-INF",
> "MANIFEST.MF");
>             writeManifest(manifestPath, "foo.bar.Baz");
>         }
>
>         // repeat for bar.jar but with manifest being written out first
>         final Path barJar = Paths.get("bar.jar");
>         try (final FileSystem zipFs =
> FileSystems.newFileSystem(URI.create("jar:" + barJar.toUri()), options)) {
>             Files.createDirectories(zipFs.getPath("META-INF"));
>             final Path manifestPath = zipFs.getPath("META-INF",
> "MANIFEST.MF");
>             // first write manifest content
>             writeManifest(manifestPath, "foo.bar.Baz");
>             // now write text file
>             writeTxtFile(zipFs.getPath("foo.txt"));
>         }
>
>         // now check the jar contents
>         final Path[] jars = new Path[] {jarPath, barJar};
>         for (final Path p : jars) {
>             try (InputStream fileInputStream = new
> FileInputStream(p.toFile());
>                 final JarInputStream jaris = new
> JarInputStream(fileInputStream);) {
>                 final Manifest manifest = jaris.getManifest();
>                 if (manifest == null) {
>                     System.err.println(p + " is missing the manifest file");
>                 } else {
>                     System.out.println(p + " has the manifest file");
>                     final String mainClass =
> manifest.getMainAttributes().getValue(Attributes.Name.MAIN_CLASS);
>                     if (!"foo.bar.Baz".equals(mainClass)) {
>                         System.err.println("Found unexpected main class
> " + mainClass + " in jar " + p);
>                     }
>                 }
>             }
>         }
>
>     }
>
>     private static void writeTxtFile(final Path filePath) throws
> IOException {
>         try (final OutputStream os = Files.newOutputStream(filePath)) {
>             final byte[] someData = new byte[]{'b', 'c', 'd'};
>             os.write(someData);
>         }
>     }
>
>     private static void writeManifest(final Path manifestPath, final
> String mainClass) throws IOException {
>         try (final OutputStream os = Files.newOutputStream(manifestPath)) {
>             final Manifest manifest = new Manifest();
>             final Attributes attributes = manifest.getMainAttributes();
>             attributes.put(Attributes.Name.MANIFEST_VERSION, "1.0");
>             attributes.put(Attributes.Name.MAIN_CLASS, mainClass);
>             manifest.write(os);
>         }
>     }
> }
>