Alternative to fatJar - modular solution?
Ioi Lam
ioi.lam at oracle.com
Sat Oct 9 06:19:27 UTC 2021
A module can indicate its main class by setting the ModuleMainClass
attribute in its module-info.class. This can be done by running with
"jar --main-class=xxx" when creating a modular JAR file.
The module system allows you to enumerate all the modules in a
ModuleLayer. For each module, you can call
java.lang.module.ModuleDescriptor::mainClass() to find out the main class.
The module system provides many other ways for inspecting the modules. E.g.,
- You can iterate over all the packages of a module with
java.lang.Module::getPackages().
- You can iterate over all the contents (classfiles, resources, etc)
using java.lang.module.ModuleReader::list().
Given the introspection support of the module system, I am not sure why
there's a need to scan all the class. But if there's a need to do that,
we can probably do it as efficient as SprintBoot can today with
non-modularized Uber-JARs (my extremely slow inefficient implementation
of uber: protocol notwithstanding).
I would go out on a limb and say that using modularized Uber-JARs can be
as efficient as using "java --module-path=modulesdir". If all the
contents of the Uber-JAR are uncompressed, the contents of the embedded
JAR files can be accessed with DirectBuffer, etc.
Note:
- I am not pro or against using Uber-JARs for modularized apps
- All I am saying is -- if this is the direction one wants to go, there
doesn't seem to be a performance barrier for doing so. And the Java core
API already has the underpinnings for a possible implementation.
Thanks
- Ioi
On 10/8/21 9:34 PM, Samuel Audet wrote:
> One problem is that frameworks like Spring Boot need to scan the
> classes that are available in a module, to get a list of the names of
> all the classes, among other things. The JDK provides no standard way
> to do that. For example, how would you implement a class like this one?
>
> https://github.com/spring-projects/spring-boot/blob/master/spring-boot-project/spring-boot-tools/spring-boot-loader-tools/src/main/java/org/springframework/boot/loader/tools/MainClassFinder.java
> <https://urldefense.com/v3/__https://github.com/spring-projects/spring-boot/blob/master/spring-boot-project/spring-boot-tools/spring-boot-loader-tools/src/main/java/org/springframework/boot/loader/tools/MainClassFinder.java__;!!ACWV5N9M2RV99hQ!YbVLiZRxgAhtRJbb20kgZBxN1NdNgoJThcrTXFWAToYem9oFPXJDIgKJQVzd6A$>
>
> Samuel
>
> On Sat, Oct 9, 2021, 10:21 Ioi Lam <ioi.lam at oracle.com
> <mailto:ioi.lam at oracle.com>> wrote:
>
> As a proof of concept, I wrote a quick-and-dirty demo that can load
> Jigsaw modules from a Uber-JAR file:
>
> https://github.com/iklam/tools/tree/main/jigsaw/uberjar
> <https://urldefense.com/v3/__https://github.com/iklam/tools/tree/main/jigsaw/uberjar__;!!ACWV5N9M2RV99hQ!YbVLiZRxgAhtRJbb20kgZBxN1NdNgoJThcrTXFWAToYem9oFPXJDIgLt6Q8uqw$>
>
> Glavo, would something like this fit your needs?
>
> I think something like this could be done outside the JDK (in
> frameworks
> like SpringBoot) or inside the JDK (as some sort of extension to
> "java
> -jar", for example).
>
> As the demo shows, we already have the core API support to enable
> this.
> I would encourage other member of the Java community to experiment
> with
> it and see if we should proceed further in this direction.
>
> Thanks
> - Ioi
>
> On 10/7/21 9:31 PM, Ioi Lam wrote:
> > I am wondering if there are 3rd party solutions that support
> loading
> > Jigsaw modules from uber jars. The JDK should have all the APIs to
> > support such a solution.
> >
> > E.g., I looked at SpringBoot, which has uber jar support, but it
> > doesn't seem to support modules ("java.lang.module" doesn't
> appear in
> > any of the source files).
> >
> > Thanks
> > - Ioi
> >
> > On 10/7/21 6:43 AM, Gregg Wonderly wrote:
> >> The URL class loader is the easiest way to solve conditional
> >> loading. In Jini, now Apache River, we’ve long used this
> mechanism
> >> to “get” the implementation of all interfaces that a remote client
> >> application needed to talk to a particular server (versioning
> makes
> >> this necessary and powerfully easy as a solution). But the
> security
> >> manager didn’t seem useful as of late and the removal of the
> security
> >> manager support for managing per jar security is a bit problematic
> >> for this kind of mobile code use in Java. Yet, this is the
> primary
> >> way that javascript works in the web browser as the mobile code
> >> interface to remote services. It really feels like Oracle and the
> >> Java team have no interest in what the desktop environment
> represents…
> >>
> >> Gregg Wonderly
> >>
> >>> On Oct 7, 2021, at 7:41 AM, Glavo <zjx001202 at gmail.com
> <mailto:zjx001202 at gmail.com>> wrote:
> >>>
> >>>> *Bandwidth optimization and rare machines.* This is interesting
> >>>> because
> >>>> it's a requirement that feels like it may be more common in
> China than
> >>>> elsewhere. I'd be keen to learn more about your bandwidth
> constraints,
> >>>> unless this is more of a theoretical concern?
> >>>
> >>> Ah, in fact, in Chinese mainland, server bandwidth is a very real
> >>> problem. In China, many websites such as GitHub and cloudflare
> cannot
> >>> provide services normally. The cost of civil broadband is low, but
> >>> commercial bandwidth is more expensive, which costs several
> times or
> >>> even more than ten times higher than other parts of the world.
> >>> Our average income level also lags behind that of developed
> countries,
> >>> so we will pay more attention to the cost of bandwidth.
> >>>
> >>> Mike Hearn <mike at plan99.net <mailto:mike at plan99.net>>
> 于2021年10月7日周四 下午7:31写道：
> >>>
> >>>> Thanks for your insightful reply, Glavo. Here are some
> thoughts. I
> >>>> should
> >>>> note that I don't work for Oracle or on OpenJDK, in case that
> wasn't
> >>>> already clear.
> >>>>
> >>>> *Forum.* Although it's logical that you ended up on this list,
> >>>> realistically the JPMS is "done" and not being worked on
> since Java
> >>>> 9. Any
> >>>> solutions or improvements have to come from the user
> community so
> >>>> it may
> >>>> make more sense to have this discussion on Reddit, or some other
> >>>> Java forum.
> >>>>
> >>>> *Alternative approach. *Given this constraint, it can make
> sense to
> >>>> think
> >>>> wider or bigger than just updating previous approaches. Would
> your
> >>>> needs be
> >>>> met or even met better by a re-imagining of Web Start, but one
> >>>> suitable for
> >>>> servers and the CLI? For example:
> >>>>
> >>>> $ alias glavos="jrun glavos-cool-app.com
> <https://urldefense.com/v3/__http://glavos-cool-app.com__;!!ACWV5N9M2RV99hQ!YbVLiZRxgAhtRJbb20kgZBxN1NdNgoJThcrTXFWAToYem9oFPXJDIgK30k-y9g$>"
> >>>> $ glavos --flag --another-flag
> >>>>
> >>>> Here an imaginary "jrun" command (re)downloads an app and
> stores it
> >>>> to a
> >>>> local cache, perhaps downloading an appropriate JVM/jlinked
> image
> >>>> alongside
> >>>> it if none is available already locally. It's given a URL but
> in a
> >>>> convenient form for typing, e.g. with assumed protocols and
> paths
> >>>> if only a
> >>>> domain name is specified. The tool would occasionally check for
> >>>> updates and
> >>>> run from the cache the rest of the time. This doesn't make
> apps into a
> >>>> single file but it tackles other problems you mention having to
> >>>> roll your
> >>>> own solutions for, like writing your own update checker and
> asking
> >>>> users to
> >>>> download the right file. Unlike tools like apt-get or brew there
> >>>> would be
> >>>> no notion of adding a repository beforehand, so for CLI / server
> >>>> apps, it
> >>>> retains its usability.
> >>>>
> >>>> For desktop apps a simple .jrun file association could be
> used to
> >>>> do the
> >>>> same thing.
> >>>>
> >>>> For building Docker images you could have:
> >>>>
> >>>> $ jrun --cache-only glavos-cool-app.com
> <https://urldefense.com/v3/__http://glavos-cool-app.com__;!!ACWV5N9M2RV99hQ!YbVLiZRxgAhtRJbb20kgZBxN1NdNgoJThcrTXFWAToYem9oFPXJDIgK30k-y9g$>
> >>>>
> >>>> which would populate a cache during the docker build, but not
> run the
> >>>> program itself.
> >>>>
> >>>> I've often wished for such a tool. At one point I built one that
> >>>> did Maven
> >>>> resolution, but it for GUI apps. Although my new venture is about
> >>>> self-updating desktop/server app packages, I've been planning an
> >>>> extension
> >>>> in this direction later because once you can distribute a
> generic
> >>>> runtime
> >>>> as a self-updating "app" you can easily bring back the JRE model
> >>>> for those
> >>>> who want it.
> >>>>
> >>>> *Jimage.* In your first mail you proposed a new kind of fat-jar
> >>>> based on
> >>>> the jimage format the modules file uses. JImage isn't a
> documented
> >>>> format,
> >>>> or rather, it's documented only in the source code, but it
> has quite a
> >>>> clever design. The upside is that it's highly optimized. The
> >>>> downsides are:
> >>>>
> >>>> 1. Write only. ZIPs have some basic support for editing but
> jimage
> >>>> doesn't. This is a pain for things like config files, where
> you may
> >>>> want to
> >>>> make specialized versions of an app by adjusting the internal
> >>>> files. It can
> >>>> be easily fixed using a classloader that checks local disk for
> >>>> resources
> >>>> first.
> >>>>
> >>>> 2. No built-in support for native code libraries. There was a
> related
> >>>> discussion of this problem a week or so ago on this list. Of
> >>>> course, JARs
> >>>> have the same problem.
> >>>>
> >>>> 3. No support for multiple versions of the same JAR in the same
> >>>> file, even
> >>>> though the core JPMS *can* support this via the
> >>>> defineModulesWithManyLoaders API, and even though this would
> be a very
> >>>> useful thing to support. Fat JARs have the same problem so
> this is
> >>>> not a
> >>>> downside compared to the status quo.
> >>>>
> >>>> 4. The format is deliberately undocumented so it can be
> changed in
> >>>> future
> >>>> JVM versions. Thus using it would actually mean cloning it,
> and/or
> >>>> rewriting parts of the code because otherwise the GPL2 might
> kick in.
> >>>>
> >>>> Overall, the downsides are not that big! The worst is the
> need to
> >>>> clone
> >>>> the format to avoid depending on JVM internals. On the other
> hand,
> >>>> ZIPs
> >>>> work well enough and don't require writing any new code except a
> >>>> little
> >>>> stub entry point that uses custom classloaders.
> >>>>
> >>>> *Bandwidth optimization and rare machines.* This is interesting
> >>>> because
> >>>> it's a requirement that feels like it may be more common in
> China than
> >>>> elsewhere. I'd be keen to learn more about your bandwidth
> constraints,
> >>>> unless this is more of a theoretical concern?
> >>>>
> >>>> You mention you actually have users on LoongArch64 for example.
> >>>> Indeed,
> >>>> the chances that non-Chinese developers will produce jlinked
> images
> >>>> for
> >>>> this CPU any time soon is very low.
> >>>>
> >>>> *Product potential.* As mentioned, I'm setting up a new venture
> >>>> that is
> >>>> starting with app distribution, and particularly distribution
> for
> >>>> the JVM
> >>>> world. JPackage is good as far as it goes, but it doesn't
> solve all
> >>>> the
> >>>> problems developers face. Given your list of target machines it
> >>>> feels like
> >>>> you're probably a commercial organization with a wide customer
> >>>> base. If
> >>>> you're in the market for better approaches please send an
> email to
> >>>> mike at hydraulic.software and maybe your needs can influence
> our product
> >>>> direction.
> >>>>
> >>>>
> >>>>
> >
>
More information about the jigsaw-dev
mailing list