Proposal: A new common Image API

[Glavo]

Hi Kevin,

Going from two implementations to three is really frustrating. But what’s
even more frustrating is
that the Java ecosystem now doesn’t have just two implementations—it has
five, six, or even more.
In addition to AWT and JavaFX, there are Android, SWT, Qt Jambi, and Skija,
along with countless other image libraries.

The problem isn’t that there are multiple implementations, but that none of
them are common enough,
so developers keep reinventing the wheel. Here’s a recent example:
jxlatte[1]. It’s a pure Java JPEG XL decoder.
To avoid depending on java.desktop, its developers once again created new
abstractions for colors and images.

The current situation is chaotic for the entire ecosystem and also sets a
high barrier for creating new libraries.
Developing a small library for image processing should be simple,
but now developers have to think carefully before starting—and might even
end up reinventing the wheel themselves.
This also creates a poor experience for users.

I’m not suggesting we deprecate all existing implementations, but I do
think a new common library is important.
This library should provide a unified facade for existing implementations
(SLF4J is a good example)
and prevent developers from having to reinvent the wheel repeatedly.

Glavo

[1]: https://github.com/Traneptora/jxlatte

On Sun, Apr 20, 2025 at 12:11 AM Kevin Rushforth <kevin.rushforth at oracle.com>
wrote:

> I agree with Phil on this. I don't see enough value in creating yet
> another image library in the JDK to justify the effort and the weight of
> the API and implementation. We would need to design, specify, implement,
> and support a new imaging API all while maintaining the existing JavaFX and
> Java2D imaging libraries. We wouldn't deprecate the existing imaging
> libraries in favor of the new one, at least not any time in the next
> several years (if ever), so we would either end up with three
> implementations (instead of two) or additional effort to redo the
> implementation BufferedImage, Raster, Image, etc, on top of the new
> library. Similarly for JavaFX's (smaller) implementation. And then there's
> Image IO, which would also need to be redone.
>
> So I am not at all optimistic about this, and I doubt this is something we
> should pursue.
>
> --Kevin
>
>
> On 4/18/2025 8:37 PM, Glavo wrote:
>
> Hi Jeremy,
>
> The purpose of my email is to:
>
>
>    1. Find out if people feel the work is worth the effort.
>    2. Find out if there is anyone willing to take the lead.
>    3. Discuss how this work will begin.
>
> I did have a draft design, but I was not an expert in the field, so I knew
> it had a lot of flaws.
> I guess if that's what people really want, then people should have their
> own ideas about it.
> So instead of presenting this flawed idea, I wanted to discuss other
> things first.
>
> Once I've determined that the work is worth the effort, but no expert has
> the free time to take the lead,
> I'll try to implement it myself and then seek guidance.
>
> What is the difference between what you’re describing and the Apache
>> Commons Imaging project itself?
>
>
> Apache Commons Imaging currently depends on AWT BufferedImage. It does not
> include the things mentioned in the email.
>
> Glavo
>
> On Fri, Apr 18, 2025 at 4:37 AM Jeremy Wood <mickleness at gmail.com> wrote:
>
>> What is the scope / ideal feature set of this project? (Can you start
>> outlining some of the main interfaces you’re envisioning? maybe in a
>> google doc?)
>>
>> What exactly are you looking for in this email thread? Are you looking
>> for resources (mostly people?) who can work on this project? Or are you
>> looking to see if OpenJDK is willing/able to maintain this API? (or
>> something else?)
>>
>> What is the difference between what you’re describing and the Apache
>> Commons Imaging project itself?
>>
>> I assume this project is strictly related to reading & writing images. So
>> it will NOT support anything analogous to:
>> 1. Graphics2D
>> 2. AffineTransform
>> 3. PerspectiveTransform
>> 4. Fonts
>> 5. MultiResolutionImage
>>
>>  - Jeremy
>>
>> obligatory xkcd reference:
>>
>> https://www.explainxkcd.com/wiki/index.php/927:_Standards
>>
>> ------ Original Message ------
>> From "Scott Palmer" <swpalmer at gmail.com>
>> To "Philip Race" <philip.race at oracle.com>
>> Cc "Glavo" <zjx001202 at gmail.com>; "openjfx-dev" <openjfx-dev at openjdk.org>;
>> client-libs-dev at openjdk.org
>> Date 4/17/2025 12:57:17 PM
>> Subject Re: Proposal: A new common Image API
>>
>> I think a common image I/O library that is not tied to a UI
>> framework makes sense and is long overdue.
>> Raster images do have a common format that encapsulates everything. We
>> essentially have this abstracted in the two UI frameworks already. At some
>> level it comes down to PixelFormats and data buffers.  There are not so
>> many of them that it is impossible to make a common abstraction for the
>> purposes of I/O that can be mapped to what is needed by the UI framework.
>> Just as JavaFX already has the SwingFxUtils for converting between AWT
>> and JavaFX formats, there can be a utility to convert between the I/O
>> library format and each UI framework's format. I would expect in most cases
>> that the raw pixel data could be shared without extra copying.
>> ImageIO is a good starting point. Remove the actual UI classes from it
>> like BufferedImage and keep plain raster representations of the data that
>> can be wrapped by the UI classes.  Under the hood the arrays or buffers of
>> raster data don't have to change,they are the important parts that the I/O
>> library needs to deal with.  Mapping the metadata (width, height, colour
>> space, pixel format, etc.) will usually be very cheap. Some cases may need
>> to run a conversion, like the example of 1-bpp black/white needing to be
>> remapped to RGB, but that that can happen in the utility layer that moves
>> the image from the Image I/O domain to the UI framework domain on a
>> case-by-case basis. Worst case is that the UI framework throws an
>> UnsupportImageFormat exception when it doesn't have code to deal with
>> raster data in a particular format.
>>
>> I'm sure it is all much harder than I suspect, but I don't think it
>> should be. :-)
>>
>> Scott
>>
>>
>> On Thu, Apr 17, 2025 at 12:10 AM Philip Race <philip.race at oracle.com>
>> wrote:
>>
>>> First, note than John Neffenger replied to this but only on openjfx-dev
>>> and the first thing I saw was the reply and couldn't see the original.
>>> After some consternation I tracked down this cross-post.
>>>
>>> Here's a link to the reply
>>> https://mail.openjdk.org/pipermail/openjfx-dev/2025-April/053616.html
>>>
>>> A fundamental problem is that all the users need to be able to produce
>>> and consume the data.
>>> So there either needs to be a module dependency (not viable) or an
>>> agreed format (are we
>>> really going to define an image format which encapsulates everything,
>>> including the multi-frame
>>> GIF support) and then everyone needs a reader and don't forget writers
>>> and they need to be able to do .. so much ..
>>>
>>> I just don't see a viable path here.
>>> And several (8 ?) years ago, I pondered some way to separate image
>>> handling from the
>>> desktop module to see if a server app could use it without pulling in
>>> AWT but the intra-package
>>> dependencies made it impossible without changes I didn't even figure out
>>> if they were possible.
>>>
>>> -phil.
>>>
>>> On 4/16/25 3:04 AM, Glavo wrote:
>>> > Currently, there are multiple different image APIs in the Java
>>> > ecosystem: AWT, JavaFX, Android, etc.
>>> > What's worse, the Android platform does not provide support for AWT,
>>> > making the Java ecosystem even more fragmented.
>>> >
>>> > There are some obvious problems with the current situation:
>>> >
>>> > * Third-party libraries that need an image API are difficult to be
>>> > universal.
>>> >   A practical example: Apache Commons Imaging has been in the alpha
>>> > stage and cannot release version 1.0.
>>> >   The main reason is that it depends on `java.awt.image`, so it
>>> > doesn't work on Android.
>>> >   We hope to solve this problem before the official release.
>>> > * Different image APIs have to repeatedly implement support for
>>> > reading the same image format (such as JPEG).
>>> >   In fact, AWT, JavaFX, and Android now each implement reading JPEG
>>> > images.
>>> >   This is a waste.
>>> >
>>> > I thought we might be able to create a new module independent of
>>> > java.desktop that provides a common abstraction for images.
>>> > It should:
>>> >
>>> > * Provides common Image and ImageProvider interfaces that can be
>>> > implemented by different providers.
>>> > * Provides a unified abstraction for colors, color spaces, pixel
>>> > formats, etc.
>>> > * Provides general and extensible image I/O support.
>>> >   Read/write support should only need to be implemented once per image
>>> > format.
>>> >   It should be bidirectionally compatible with `javax.imageio`:
>>> >   The implementation of either API can be accessed through the other
>>> API.
>>> >
>>> > I want to know if this is an idea worth putting into practice?
>>> > I'm not an expert in this field, so I'm worried about creating designs
>>> > with many flaws.
>>> > Therefore, I haven't attempted to implement it yet.
>>> > If anyone is willing to implement it, I'd like to help.
>>> >
>>> > I had sent an email a few days ago but no one responded, so I
>>> > re-edited it and sent this one.
>>> >
>>> > Glavo
>>> >
>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/openjfx-dev/attachments/20250420/b82034ed/attachment-0001.htm>