Alternative to fatJar - modular solution?
Samuel Audet
samuel.audet at gmail.com
Sat Oct 9 05:44:54 UTC 2021
Well, we can still use ZipInputStream just to list files from a JAR, but
it's not exactly efficient. Would anyone want to make class scanning 10x
slower just to support modules? Probably not
Samuel
On Sat, Oct 9, 2021, 13:34 Samuel Audet <samuel.audet at gmail.com> wrote:
> One problem is that frameworks like Spring Boot need to scan the classes
> that are available in a module, to get a list of the names of all the
> classes, among other things. The JDK provides no standard way to do that.
> For example, how would you implement a class like this one?
>
>
> https://github.com/spring-projects/spring-boot/blob/master/spring-boot-project/spring-boot-tools/spring-boot-loader-tools/src/main/java/org/springframework/boot/loader/tools/MainClassFinder.java
>
> Samuel
>
> On Sat, Oct 9, 2021, 10:21 Ioi Lam <ioi.lam at oracle.com> wrote:
>
>> As a proof of concept, I wrote a quick-and-dirty demo that can load
>> Jigsaw modules from a Uber-JAR file:
>>
>> https://github.com/iklam/tools/tree/main/jigsaw/uberjar
>>
>> Glavo, would something like this fit your needs?
>>
>> I think something like this could be done outside the JDK (in frameworks
>> like SpringBoot) or inside the JDK (as some sort of extension to "java
>> -jar", for example).
>>
>> As the demo shows, we already have the core API support to enable this.
>> I would encourage other member of the Java community to experiment with
>> it and see if we should proceed further in this direction.
>>
>> Thanks
>> - Ioi
>>
>> On 10/7/21 9:31 PM, Ioi Lam wrote:
>> > I am wondering if there are 3rd party solutions that support loading
>> > Jigsaw modules from uber jars. The JDK should have all the APIs to
>> > support such a solution.
>> >
>> > E.g., I looked at SpringBoot, which has uber jar support, but it
>> > doesn't seem to support modules ("java.lang.module" doesn't appear in
>> > any of the source files).
>> >
>> > Thanks
>> > - Ioi
>> >
>> > On 10/7/21 6:43 AM, Gregg Wonderly wrote:
>> >> The URL class loader is the easiest way to solve conditional
>> >> loading. In Jini, now Apache River, we’ve long used this mechanism
>> >> to “get” the implementation of all interfaces that a remote client
>> >> application needed to talk to a particular server (versioning makes
>> >> this necessary and powerfully easy as a solution). But the security
>> >> manager didn’t seem useful as of late and the removal of the security
>> >> manager support for managing per jar security is a bit problematic
>> >> for this kind of mobile code use in Java. Yet, this is the primary
>> >> way that javascript works in the web browser as the mobile code
>> >> interface to remote services. It really feels like Oracle and the
>> >> Java team have no interest in what the desktop environment represents…
>> >>
>> >> Gregg Wonderly
>> >>
>> >>> On Oct 7, 2021, at 7:41 AM, Glavo <zjx001202 at gmail.com> wrote:
>> >>>
>> >>>> *Bandwidth optimization and rare machines.* This is interesting
>> >>>> because
>> >>>> it's a requirement that feels like it may be more common in China
>> than
>> >>>> elsewhere. I'd be keen to learn more about your bandwidth
>> constraints,
>> >>>> unless this is more of a theoretical concern?
>> >>>
>> >>> Ah, in fact, in Chinese mainland, server bandwidth is a very real
>> >>> problem. In China, many websites such as GitHub and cloudflare cannot
>> >>> provide services normally. The cost of civil broadband is low, but
>> >>> commercial bandwidth is more expensive, which costs several times or
>> >>> even more than ten times higher than other parts of the world.
>> >>> Our average income level also lags behind that of developed countries,
>> >>> so we will pay more attention to the cost of bandwidth.
>> >>>
>> >>> Mike Hearn <mike at plan99.net> 于2021年10月7日周四 下午7:31写道:
>> >>>
>> >>>> Thanks for your insightful reply, Glavo. Here are some thoughts. I
>> >>>> should
>> >>>> note that I don't work for Oracle or on OpenJDK, in case that wasn't
>> >>>> already clear.
>> >>>>
>> >>>> *Forum.* Although it's logical that you ended up on this list,
>> >>>> realistically the JPMS is "done" and not being worked on since Java
>> >>>> 9. Any
>> >>>> solutions or improvements have to come from the user community so
>> >>>> it may
>> >>>> make more sense to have this discussion on Reddit, or some other
>> >>>> Java forum.
>> >>>>
>> >>>> *Alternative approach. *Given this constraint, it can make sense to
>> >>>> think
>> >>>> wider or bigger than just updating previous approaches. Would your
>> >>>> needs be
>> >>>> met or even met better by a re-imagining of Web Start, but one
>> >>>> suitable for
>> >>>> servers and the CLI? For example:
>> >>>>
>> >>>> $ alias glavos="jrun glavos-cool-app.com"
>> >>>> $ glavos --flag --another-flag
>> >>>>
>> >>>> Here an imaginary "jrun" command (re)downloads an app and stores it
>> >>>> to a
>> >>>> local cache, perhaps downloading an appropriate JVM/jlinked image
>> >>>> alongside
>> >>>> it if none is available already locally. It's given a URL but in a
>> >>>> convenient form for typing, e.g. with assumed protocols and paths
>> >>>> if only a
>> >>>> domain name is specified. The tool would occasionally check for
>> >>>> updates and
>> >>>> run from the cache the rest of the time. This doesn't make apps into
>> a
>> >>>> single file but it tackles other problems you mention having to
>> >>>> roll your
>> >>>> own solutions for, like writing your own update checker and asking
>> >>>> users to
>> >>>> download the right file. Unlike tools like apt-get or brew there
>> >>>> would be
>> >>>> no notion of adding a repository beforehand, so for CLI / server
>> >>>> apps, it
>> >>>> retains its usability.
>> >>>>
>> >>>> For desktop apps a simple .jrun file association could be used to
>> >>>> do the
>> >>>> same thing.
>> >>>>
>> >>>> For building Docker images you could have:
>> >>>>
>> >>>> $ jrun --cache-only glavos-cool-app.com
>> >>>>
>> >>>> which would populate a cache during the docker build, but not run the
>> >>>> program itself.
>> >>>>
>> >>>> I've often wished for such a tool. At one point I built one that
>> >>>> did Maven
>> >>>> resolution, but it for GUI apps. Although my new venture is about
>> >>>> self-updating desktop/server app packages, I've been planning an
>> >>>> extension
>> >>>> in this direction later because once you can distribute a generic
>> >>>> runtime
>> >>>> as a self-updating "app" you can easily bring back the JRE model
>> >>>> for those
>> >>>> who want it.
>> >>>>
>> >>>> *Jimage.* In your first mail you proposed a new kind of fat-jar
>> >>>> based on
>> >>>> the jimage format the modules file uses. JImage isn't a documented
>> >>>> format,
>> >>>> or rather, it's documented only in the source code, but it has quite
>> a
>> >>>> clever design. The upside is that it's highly optimized. The
>> >>>> downsides are:
>> >>>>
>> >>>> 1. Write only. ZIPs have some basic support for editing but jimage
>> >>>> doesn't. This is a pain for things like config files, where you may
>> >>>> want to
>> >>>> make specialized versions of an app by adjusting the internal
>> >>>> files. It can
>> >>>> be easily fixed using a classloader that checks local disk for
>> >>>> resources
>> >>>> first.
>> >>>>
>> >>>> 2. No built-in support for native code libraries. There was a related
>> >>>> discussion of this problem a week or so ago on this list. Of
>> >>>> course, JARs
>> >>>> have the same problem.
>> >>>>
>> >>>> 3. No support for multiple versions of the same JAR in the same
>> >>>> file, even
>> >>>> though the core JPMS *can* support this via the
>> >>>> defineModulesWithManyLoaders API, and even though this would be a
>> very
>> >>>> useful thing to support. Fat JARs have the same problem so this is
>> >>>> not a
>> >>>> downside compared to the status quo.
>> >>>>
>> >>>> 4. The format is deliberately undocumented so it can be changed in
>> >>>> future
>> >>>> JVM versions. Thus using it would actually mean cloning it, and/or
>> >>>> rewriting parts of the code because otherwise the GPL2 might kick in.
>> >>>>
>> >>>> Overall, the downsides are not that big! The worst is the need to
>> >>>> clone
>> >>>> the format to avoid depending on JVM internals. On the other hand,
>> >>>> ZIPs
>> >>>> work well enough and don't require writing any new code except a
>> >>>> little
>> >>>> stub entry point that uses custom classloaders.
>> >>>>
>> >>>> *Bandwidth optimization and rare machines.* This is interesting
>> >>>> because
>> >>>> it's a requirement that feels like it may be more common in China
>> than
>> >>>> elsewhere. I'd be keen to learn more about your bandwidth
>> constraints,
>> >>>> unless this is more of a theoretical concern?
>> >>>>
>> >>>> You mention you actually have users on LoongArch64 for example.
>> >>>> Indeed,
>> >>>> the chances that non-Chinese developers will produce jlinked images
>> >>>> for
>> >>>> this CPU any time soon is very low.
>> >>>>
>> >>>> *Product potential.* As mentioned, I'm setting up a new venture
>> >>>> that is
>> >>>> starting with app distribution, and particularly distribution for
>> >>>> the JVM
>> >>>> world. JPackage is good as far as it goes, but it doesn't solve all
>> >>>> the
>> >>>> problems developers face. Given your list of target machines it
>> >>>> feels like
>> >>>> you're probably a commercial organization with a wide customer
>> >>>> base. If
>> >>>> you're in the market for better approaches please send an email to
>> >>>> mike at hydraulic.software and maybe your needs can influence our
>> product
>> >>>> direction.
>> >>>>
>> >>>>
>> >>>>
>> >
>>
>>
More information about the jigsaw-dev
mailing list