FINAL PROPOSAL: jrawio - Image I/O codecs for "camera raw" formats
Fabrizio Giudici
fabrizio.giudici at tidalwave.it
Sun Mar 2 08:52:46 PST 2008
FINAL PROPOSAL
Provision of jrawio, a set of Image I/O codecs for "camera raw"
formats, suitable for inclusion in OpenJDK
Fabrizio Giudici
fabrizio.giudici(at)tidalwave.it
1. Summary
----------
The Java Image I/O API (javax.imageio) is a specific API of the JDK
focused on image input/output. It basically provides a plugin-based
set of classes for reading and writing images from/to a stream (both
raster and metadata operations are supported). In the standard Java
runtime a number of plugins for the most common file formats are
available (e.g. JPEG, PNG, BMP), others are in the optional package
JAI-Image I/O provided by Sun (e.g. TIFF), others are available as
third-parties' plugins.
With the advent of professional digital camera bodies, manufacturers
such as Nikon, Canon, Minolta Pentax and others have introduced a
number of proprietary formats, commonly named "camera raw" formats
(NEF, CRW, CR2, MRW, etc...) [1]. In this scenario, also Adobe
developed its own format, Adobe Digital Negative (DNG), which has got
an open specification. All "Camera raw" formats share the peculiarity
of not containing an image ready-to-be-shown, but just a "dump" of
the raw data from the digital sensor with a set of annotations about
the camera settings at the moment of the shoot. This makes them a
sort of "digital negative", as photographers can manipulate
("develop") them in post-production with a much wider gamma of
possibilities than JPEG or TIFF would offer; it also makes it easier
to implement "non-destructive processing" (the capability of undoing
a developing step without compromising the original bits).
jrawio [2,3] is an open source implementation of a set of read-only
plugins for supporting these formats.
jrawio is an original development. At the moment of writing there are
no other known Image I/O-compliant plugins for "camera raw" formats.
jrawio only uses original code (of course integrated with the Image I/
O API, which implies that jrawio implements some of the interfaces
defined by Image I/O). jrawio is partially inspired by dcraw.c by
Dave Coffin [4] as most of the "camera raw" formats are undocumented
(DNG being a notable exception) and dcraw.c is the only available
"documentation" for the community. An older dismissed project by
Fabrizio Giudici, NEFio [9], was just the initial version of jrawio
with a different name as it only focused on the NEF format.
The goal of this proposal is to complete the work required for making
jrawio a part of the OpenJDK.
This project is a single-developer submission. Fabrizio Giudici is in
charge for the rules of the Open JDK Challenge.
2. Need for jrawio
------------------
While most of “camera raw” formats are a variant of TIFF, they can't
be properly handled by the existing TIFF Image I/O plugin, because:
1) it can't understand the proprietary extensions, both in metadata
and raster layout, also because often non-standard compression
schemata are used
2) as all the other Image I/O plugins, it doesn't perform any post-
processing on the raster data
The typical post-processing required by “camera raw” formats is
composed by:
1) demosaicing
2) noise reduction
3) boosting of R-G-B channels
4) applying a color profile
5) sharpening
3. Reasons for having jrawio in the OpenJDK
-------------------------------------------
We believe that support for “camera raw” formats should be present in
the OpenJDK, and not as a separate extension, since these formats are
more and more used by advanced amateurs and professional
photographers in place of JPEG and the ubiquitous availability of
media codecs out-of-the-box is important for Java to be able to
compete with other technologies for the desktop such Adobe Air. Also,
a small but specific part of the post-processing (the demosaicing
step) could benefit from a small native accelerator, whose presence
in the Java runtime would make easier the life for Applet and JavaFX
developers. Last but not least, many communities of photographers
(see e.g. OpenRAW [10]) have publicly stressed the importance of
having open source implementations of software for managing "camera
raw" formats; the presence of out-of-the-box codecs would make Java
more friendly and appealing for developers targeting the imaging
segment.
4. Structure of jrawio
----------------------
jrawio is composed by two parts:
a) the javax.imageio plugin which only delivers the capability of
reading metadata, thumbnails and "raw" bits out of the stream;
b) reajent, the post-processor that "develops" the raw bits into a
viewable picture, implementing the typical operations described in
the above section. This is an optional component, as applications
could decide to provide their own implementation of this part
(providing special "recipes" for developing "camera raw" formats is
typical of this context).
5. Previous work
----------------
Work on jrawio and reajent has already started [2,3] and a 1.0.RC5
has been delivered which is reasonably stable and suitable for
experimental use, but not for production. Since 2007 the focus has
been on the setup of a public test suite - some required work and
technical problems about tests has slowed down the progress and
therefore there have been only a few unofficial releases in the
latest year.
The submitted project represents the continuation of that development.
At the moment, the metadata and raster I/O parts of jrawio support
the following formats: CR2, CRW, DNG, MRW, NEF, PEF, SRF, but only
for a subset of the existing camera models (each camera model is
likely to use a variant of the main format). It is estimated that the
current codebase contains about 80% of the needed classes for those
formats, that should be complete at 70% (as per the number of camera
models existing at the moment of writing). Support for other formats
(such as Kodak DCR) should be added, but it's difficult to estimate
the effort for them without prior knowledge of their structure.
Unit tests exist for about 10% of the codebase, while there are
integration tests over a set of around 1,000 image files verifying
that jrawio correctly reads rasters, thumbnails and metadata. Beyond
the already mentioned technical problems specific of the test code, a
licensing problem exists about the test set since it's mainly
composed of images available on the Web, with no permission about
redistribution. The "Imaging Test Set Repository" initiative [5]
should give a solution to this problem.
The jrawio and reajent implementations are licensed under the Apache
Software License 2.0. We are open to change it as needed for
compliance with the OpenJDK licensing.
If this proposal is selected, and proceeds to completion, the entire
project will be made available under the SCA (both the parts already
coded, and the additions made since).
6. Milestones
-------------
Development milestones shall ultimately be determined by the working
group. However the outline plan is:
1) Provide tests and appropriate bug fixes to verify the full
compliance with the javax.imageio APIs
2) Add support for all the camera models using the currently
supported formats
3) Provide a small extension mechanism (probably based on META-INF/
services) so support for new camera models requiring specific code
can be just dropped in as a classpath extension. This is needed since
new camera models using "camera raw" formats are released much more
frequently than JDK updates and application developers typically want
to stay up to date with the shortest latence.
4) Add support for new "camera raw" formats (e.g. Kodak's DCR) as
long as their structure is understood.
7. Deliverables
---------------
The deliverables are:
1) The source code of jrawio (both the jrawio plugin and the reajent
component)
2) Tests with 90%+ coverage.
3) Source code documentation in the form of javadoc suitable for
inclusion in the JDK.
Any API extension for dropping in custom-specific reajent
replacements does not form part of the deliverables of this proposal,
since it won't be possible to complete this task by the August
deadline of the OpenJDK Innovation Grant Challenge (considering that
it would require a prior submission to JSR etc...). By that time
programmers will be able to just disable reajent (by means of the
existing javax.imageio API) and will manually post-process the raster
at application level.
8. Dependencies on Sun
----------------------
There are no known dependencies on Sun - the project just supports
javax.imageio and will be kept up to date with the evolution of those
APIs. While jrawio contains some TIFF-related parts, it is completely
independent of the JAI-Image I/O TIFF plugin, both at compilation and
at runtime stages.
9. Developers
-------------
Fabrizio Giudici has been working with Java since 1996, from J2ME to
J2EE. Fabrizio has been the designer and the project leader in a
number of industrial projects including banking services, support
systems for telecommunications and a real-time distribution
middleware for telemetry data in Formula One racings. He currently
works for his own company, Tidalwave s.a.s. [8], as a Senior
Architect, Mentor and Technical Instructor. Has has developed in Open
Source since 2003 as the author of jrawio, Mistral [7] and blueMarine
[6]. He is a Member of the NetBeans Dream Team since 2007.
[1] http://en.wikipedia.org/wiki/Raw_image_format
[2] http://jrawio.dev.java.net
[3] http://jrawio.tidalwave.it
[4] http://cybercom.net/~dcoffin/dcraw/
[5] https://imaging.dev.java.net/TestSetRepository.html
[6] http://bluemarine.tidalwave.it
[7] http://mistral.tidalwave.it
[8] http://www.tidalwave.it
[9] http://sourceforge.net/projects/nefio
[10] http://www.openraw.org
--
Fabrizio Giudici, Ph.D. - Java Architect, Project Manager
Tidalwave s.a.s. - "We make Java work. Everywhere."
weblogs.java.net/blog/fabriziogiudici - www.tidalwave.it/blog
Fabrizio.Giudici at tidalwave.it - mobile: +39 348.150.6941
More information about the challenge-discuss
mailing list