Alternative to fatJar - modular solution?

Ioi Lam ioi.lam at oracle.com
Thu Oct 14 17:23:23 UTC 2021


On 10/14/21 9:49 AM, Glavo wrote:
>
>     There's no requirements that a module must be stored in a JAR file. In
>     fact, your program will not work if it was packaged into an image
>     produced by jlink. That's why we have the ModuleReader::list() API.
>
>
> I understand this, but it is not uncommon for code that has made such
> assumptions. Maintaining this assumption can reduce some pain.
>
> Take my program as an example. It is a plugin. I hope it can be used
> in a modular way, but I'm sure no one will use it for jlink in this
> century. The module API does solve the problem, but I will avoid using
> it because I am careful to maintain compatibility with Java 8, whether
> I call these APIs with reflection or multi jar, the build or test process
> becomes more complex. If the old API still works, I don't want to use
> the new API.

It's doable with JDK8 API. The codesource returned by my prototype looks 
like this:

jar:file:///tmp/apps/super-launcher.jar!/modules/com.lib

Here's an example for listing its contents using the FileSystems API:

import java.io.*;
import java.net.*;
import java.nio.file.*;
import java.util.*;
import java.util.stream.*;

public class T {
     public static void main(String argv[]) throws Exception {
         Map<String, String> env = new HashMap<>();
         URI uri = 
URI.create("jar:file:///tmp/apps/super-launcher.jar!/modules/com.lib");;
         FileSystem zipfs = FileSystems.newFileSystem(uri, env);
         Path rootPath = zipfs.getPath("/modules/com.lib");
         Stream<Path> stream = Files.walk(rootPath);
         stream.forEach(p -> System.out.println(p));
     }
}

$ /jdk/official/jdk8_b132/linux_amd64/bin/java -cp . T
/modules/com.lib
/modules/com.lib/module-info.class
/modules/com.lib/com/
/modules/com.lib/com/lib/
/modules/com.lib/com/lib/libresource.txt
/modules/com.lib/com/lib/Lib.class
/modules/com.lib/META-INF/
/modules/com.lib/META-INF/MANIFEST.MF

Thanks
- Ioi

>
>      Could you explain what the actual scenario is? Is it for patching
>     the contents of a module (similar to --patch-module)?
>
>
> Ah, these are some strange use cases. They splice an exe file and a jar
> file together so that the file can be executed directly and is a jar
> file. It also needs to find its own location and read the contents of
> exe from its own head. I think this is a strange and rare use case.
> I vaguely remember several use cases that assume their own jar state,
> but I can't recall it for a moment, so I casually use it as an example.
>
> Ioi Lam <ioi.lam at oracle.com <mailto:ioi.lam at oracle.com>> 
> 于2021年10月15日周五 上午12:07写道:
>
>     On 10/14/21 8:29 AM, Glavo wrote:
>     >
>     >     In fact, I don't understand why people started packing JAR files
>     >     inside
>     >     JAR files. Maybe there were some esoteric reasons (related to
>     >     Class-Path: attribute in manifest files???).
>     >
>     >
>     >  Sometimes it's necessary to keep jars intact and distribute them as
>     > they are. In fact, a program I just developed today is not
>     compatible
>     > with your solution: It uses
>     > cls.getProtectionDomain().getCodeSource().getLocation()
>     > find its place, create a zip file system and traverse some of
>     its folders.
>     >
>
>     Since we are discussing about a solution for storing modules in a
>     single
>     file, there's an API to list all the contents of a module --
>     java.lang.module.ModuleReader::list().
>
>     Here's an example:
>
>         import java.lang.module.*;
>
>         public class FindAllInModule {
>              public static void main(String args[]) throws Exception {
>                  Module module = java.lang.Object.class.getModule();
>                  ModuleReference mref = module.getLayer().configuration()
>                          .findModule(module.getName())
>                          .orElseThrow(() -> new RuntimeException())
>                          .reference();
>                  try (ModuleReader reader = mref.open()) {
>                       reader.list().forEach(n -> System.out.println(n));
>                  }
>              }
>         }
>
>
>     There's no requirements that a module must be stored in a JAR
>     file. In
>     fact, your program will not work if it was packaged into an image
>     produced by jlink. That's why we have the ModuleReader::list() API.
>
>     > We also have some strange use cases that require additional data
>     to be
>     > appended before the jar content. Dismantling the jar will
>     destroy the
>     > data.
>
>     Could you explain what the actual scenario is? Is it for patching the
>     contents of a module (similar to --patch-module)?
>
>
>     Thanks
>     - Ioi
>
>
>
>     > Ioi Lam <ioi.lam at oracle.com <mailto:ioi.lam at oracle.com>
>     <mailto:ioi.lam at oracle.com <mailto:ioi.lam at oracle.com>>>
>     > 于2021年10月14日周四 上午8:57写道:
>     >
>     >     Hi Glavo,
>     >
>     >     I have simplified my prototype so now there's no need to
>     implement
>     >     new
>     >     URL handlers.
>     >
>     > https://github.com/iklam/tools/tree/main/jigsaw/uberjar
>     <https://urldefense.com/v3/__https://github.com/iklam/tools/tree/main/jigsaw/uberjar__;!!ACWV5N9M2RV99hQ!cYFbCdwkXUhUsnngS65ZZA8kGcFG71HOlxe0fFh73VvLG5Y6LHCbQrk4NqVAJA$>
>     >   
>      <https://urldefense.com/v3/__https://github.com/iklam/tools/tree/main/jigsaw/uberjar__;!!ACWV5N9M2RV99hQ!e5b75N_Cpd4IjSBjjO1rN9cnWFTiv-dPnb8qKrG9xrFoL9LH9NDBuNVoO-O7nQ$
>     <https://urldefense.com/v3/__https://github.com/iklam/tools/tree/main/jigsaw/uberjar__;!!ACWV5N9M2RV99hQ!e5b75N_Cpd4IjSBjjO1rN9cnWFTiv-dPnb8qKrG9xrFoL9LH9NDBuNVoO-O7nQ$>>
>     >
>     >     Please see the "Super-JAR Demo" section.
>     >
>     >     The new demo uses standard features supported by the JDK's
>     built-in
>     >     "jar:" URL handler. The only difference with my previous demo is
>     >     that we
>     >     store the "exploded" version of the modules. I.e., the JAR
>     file looks
>     >     like this:
>     >
>     >          modules/com.lib/com/lib/Lib.class
>     >          modules/com.lib/module-info.class
>     >          ...
>     >          modules/com.simple/com/simple/Simple.class
>     >          modules/com.simple/com/simple/Simple$Foo.class
>     >          modules/com.simple/module-info.class
>     >
>     >     All the modules are loaded from the /modules directories in the
>     >     JAR file.
>     >
>     >     The URI for a class looks like this:
>     >
>     >
>      jar:file:///tmp/apps/super-launcher.jar!/modules/com.lib/com/lib/Lib.class
>     >
>     >     For modularized apps, I think this is a much better approach
>     than the
>     >     traditional Uber-JARs that store JAR files inside a JAR file,
>     >     which will
>     >     require more complex decompression.
>     >
>     >     In fact, I don't understand why people started packing JAR files
>     >     inside
>     >     JAR files. Maybe there were some esoteric reasons (related to
>     >     Class-Path: attribute in manifest files???).
>     >
>     >     But, whatever reason they had would not apply to a modular
>     >     application,
>     >     where every component is already in a Jigsaw module. Packing the
>     >     exploded image into a JAR file will be good enough.
>     >
>     >     **********************
>     >
>     >     Going forward, I would suggest --
>     >
>     >     [1] Frameworks such as SpringBoot can consider the idea in this
>     >     demo for
>     >     a possible solution for packaging modules
>     >
>     >     [2] For the JDK, we should investigate supporting a single-file
>     >     packaging format for modules. E.g. extend the --module-path
>     >     command-line
>     >     option to support modules that are stored in a single file
>     (either
>     >     a JAR
>     >     file or an image file produced by jlink).
>     >
>     >          java --module-path=super-jar.jar -m com.simple
>     >     or
>     >          java --module-path=super-jar.jar -m com.simple
>     >
>     >     Or even this (with appropriate attributes in the JAR manifest):
>     >
>     >         java -jar super-jar.jar
>     >
>     >     I believe [2] is doable as the underpinning support is
>     already in the
>     >     JDK. We need to decide what format to support, how to
>     specify the
>     >     location of the modules directory inside a JAR file, etc.
>     >
>     >     As always, since the Oracle Java team has limited resources,
>     >     participation from the Java community is very much
>     appreciated and
>     >     encouraged :-)
>     >
>     >     Thanks
>     >     - Ioi
>     >
>     >
>     >
>     >     On 10/11/21 3:48 PM, Glavo wrote:
>     >     > I mistakenly believe that the implementation of the filesystem
>     >     corresponds
>     >     > exactly to the URL. The problem I really want to express
>     is that JDK
>     >     > does not support URLs of nested jar file systems. It seems
>     that this
>     >     > problem still exists in JDK 17. To make matters worse, we can
>     >     use toUri()
>     >     > to convert the path of the file in the nested jar into a URI,
>     >     but this
>     >     > URI is neither accepted by Paths.get
>     >     (java.lang.IllegalArgumentException:
>     >     > URI does not contain path info ex.
>     jar:file:/c:/foo.zip!/BAR) nor
>     >     > converted into a URL (java.net
>     <https://urldefense.com/v3/__http://java.net__;!!ACWV5N9M2RV99hQ!cYFbCdwkXUhUsnngS65ZZA8kGcFG71HOlxe0fFh73VvLG5Y6LHCbQrnvfGKdSg$>
>     >   
>      <https://urldefense.com/v3/__http://java.net__;!!ACWV5N9M2RV99hQ!e5b75N_Cpd4IjSBjjO1rN9cnWFTiv-dPnb8qKrG9xrFoL9LH9NDBuNXMJHaheg$
>     <https://urldefense.com/v3/__http://java.net__;!!ACWV5N9M2RV99hQ!e5b75N_Cpd4IjSBjjO1rN9cnWFTiv-dPnb8qKrG9xrFoL9LH9NDBuNXMJHaheg$>>.MalformedURLException:
>     >     Nested JAR URLs
>     >     > are not supported). Is this a bug or an expected behavior?
>     >     >
>     >     > Alan Bateman <Alan.Bateman at oracle.com
>     <mailto:Alan.Bateman at oracle.com>
>     >     <mailto:Alan.Bateman at oracle.com
>     <mailto:Alan.Bateman at oracle.com>>> 于2021年10月12日周二 上午2:58写道:
>     >     >
>     >     >> On 11/10/2021 15:09, Glavo wrote:
>     >     >>> I think this is a great prototype. Based on it, I think such
>     >     requirements
>     >     >>> can also be realized by enhancing jar in these aspects:
>     >     >>>
>     >     >>>     1. Nested jar file system (The ujar file system seems
>     >     unnecessary.
>     >     >>>        I never understand why jar file systems cannot be
>     nested.)
>     >     >> This was fixed in JDK 12, are you seeing issues with
>     release recent
>     >     >> releases? If so then would it be possible to submit a bug
>     with
>     >     a test
>     >     >> case or bring the issue to core-libs-dev?
>     >     >>
>     >     >> -Alan
>     >     >>
>     >
>



More information about the jigsaw-dev mailing list