DRAFT PROPOSAL: Implement XRender pipeline for Java2D

[Clemens Eisserer]

Hi again,

Attention: I am from Austria, as long as Austria is not in the list of
countries allowed to participiate I may not work on this :-/

I reworked my previous proposal and focused only on the XRender
backend, because thats the core functionality the old proposal focused
on. I think its better to get this done right instead of many
half-done ("ready for showcase") features.
The adaption of GTK is almost done and the adoption of the theme
should not be a lot of work, so these are things I plan to do in my
spare-spare time.

As last time it would be great if you could review my proposal, I
would be really happy about comments and critics.

1.) Introduction
Since its introduction, Java2D supports very advanced rendering
operations, although only very basic operations were accelerated by
hardware, because the underlying APIs were very poor.
Because visual appearance is getting more and more attention these
days those operations are also used for performance critical
applications like games, graphical user interfaces (e.g. nimbus) or
animations – leading to poor performance for graphical intensive
applications.


Modern 3D graphic APIs, like OpenGL or Direct3D, allow to map most of
Java2D's functionality without the need for expensive read-backs
and/or software rendering.
To make that functionality available to the Java platform two new
„pipelines" where created:
The OpenGL pipeline for non Windows platforms, and the Direct3D
pipeline for the Microsoft Windows operating systems only.


However OpenGL on Unix in general has some disadvantages compared to
X11 for typical desktop applications, because of poor drivers its
often not possible to use that new functionality and Java falls back
to the default X11 rendering pipeline.
An Xrender backend for Java2D would provide almost the same features
as the existing OpenGL pipeline, however would run on a much larger
set of hardware and would integrate better into existing X11 based
desktops.

With the Xrender backend developers and users could rely on almost the
same set of functionality accelerated by default on all platforms
supported currently by OpenJDK.


2.) Status Quo
The existing X11 pipeline used on Unix systems is only capable of
accelerating basic operations and falls back to software rendering
even for often used operations like antialiasing (even for text!) or
translucent images often causing expensive VRAM readbacks.
Because of X11's very limited rendering capabilities Xrender was
introduced back in 2001, and after a long delay first drivers are
available which accelerate large areas of Xrender.

XRender allows to accelerate most of Java2D's functionality, and it is
considered to be the de-facto standard for advanced 2D graphics,
already used by a large set of libraries and toolkits like Cairo,
GTK+, QT4 and KDE4.
Because of the broad and extensive use of Xrender there is currently
major work ongoing to modify drivers for better Xrender performance.


There is already an OpenGL backend for Java2D which is about as
powerful as its Direct3D counterpart on Windows.
A small overview about the advantages/disadvantages Xrender has
compared to OpenGL:


+ Lower per-primitive overhead:
Because Xrender is compared to OpenGL a lot simpler, there is less
work for the accelerating driver to do for each primitive rendering
operation. This is especially important for complex shapes which are
transformed to many rectangles which then are passed to the underlying
graphic API.
To make it short: Because Xrender focuses on only a small set of
„simple" operations, driver implementers can focus on making those few
operations as fast as possible.


+ Support for higher-level primitives:
Xrender has support for client-side font rendering, which means glyphs
which are cached in vram can be shared across different applications.
For OpenGL each application has to upload its glyphs into a private
texture wasting large amounts of Vram.
Because it was designed with 2D graphics in mind many features can be
directly mapped to Xrender without the need of complex shaders.


+ Reliable AntiAliasing:
Because Xrender specifies exactly how antialiasing has to be done,
Java2D can rely on Xrender's antialiasing capabilities. The Direct3D
and OpenGL pipelines render a alpha-mask in software, upload it to
vram and only let the GPU only do the final composition step.
Because of this, the amount of data transferred between the Xserver
and the Java application would be reduced a lot, making antialising
even usable over slow remote connections.


+ Better driver support:
On Linux only proprietary OpenGL implementations provide enough
functionality to let the OpenGL pipeline run. However even those
proprietary drivers were/are buggy in the code-paths exercised by the
OpenGL pipeline.
For Xrender there are already many optimized drivers, both open-source
(Intel, ATI, ...) and proprietary (Nvidia). Because of the high
adaption rate of Xrender lately (KDE4, Cairo, GTK+, ...) it can be
expected that soon more and more drivers will support Xrender
acceleration.

+ Better desktop integration:
Xrender applications are „native" X11 applications, and therefor work
well with composition managers. They also get visuals optimized for
2D, and do not waste Vram with additional buffers they get, although
they are not used.

+ Better fallback behaviour:
If a driver does not accalerate some operations, its much more likely
that the operation still can be done „fast enough", because the whole
data-moving between the X-Server and Java is removed.
This also removes the burden of lowest-level optimization in the
Java2D software loops – this is now done by the X-Server.

+ Better network transparency:
Although OpenGL can also be made network-transparent via GLX, the
OpenGL API was not designed to perform especially well in a network
environment.

X11/XRender relies on the Client-Server approach and therefor is
optimized for exactly this case.


- Not programmable:
A disadvantage of Xrender is that its not as flexible as OpenGL.
OpenGL allows to use shaders to implement features not provided by
fixed functionality, the only solution for a Xrender based backend is
to fall back to software rendering.

Unlike on Windows where Microsoft starting with Windows Vista
more or less deprecated all interfaces except Direct3D, and therefor
forces GPU/driver manufacturers to create drivers well suited also for
2D rendering, on Unix good
OpenGL drivers are still rare and only proprietary drivers are till
now able to run the OpenGL pipeline.
However those proprietary drivers are often optimized for heavy-weight 3D
applications like games and are not really suited for rendering
light-weight simple 2D applications.




3.) Goals
The goal of this proposal is to make use of Xrender for accelerating
advanced Java2D functionality.
This would allow multimedia content to take advantage of modern
hardware, and would bring JDK6 Update10 graphic features also to
X11/Unix based platforms.

Some of the most important enhancements compared to the X11 pipeline include:
* Accelerated text rendering for normal, antialiased and subpixel
antialiased text.
* Support for translucent images
* Composition (colors with alpha values, ...)
* Antialiasing


4.) Approach
XRender has a lot in common with OpenGL/Direct3D and is a very
low-level API which supports only a limited number of drawing
primitives but it is very flexible which type of operations it
operates on those primitives. Therefor the approach would be to create
a brand new backend based on the shared code developed for the
OpenGL/Direct3D pipelines in JDK6-u10.
The advantage of this approach would be a „clean start", as well as a
lot shared code between OpenGL/Direct3D/XRender backends.
The downside is that some highly optimized primitives which are
supported by X11's core drawing (Arcs, Elipses, ..) maybe perform a
little bit less well. Furthermore new code needs a lot of testing.



5.) Milestones
5.1) Create a skeleton which is able to use the shared OpenGL/Direct3D code
5.2) Implement only enough to functionality to make
software-rendering-loops work.
5.3) Extend the created skeleton, accelerate basic operations -> Make
typical Swing/Ocean applications work well.
5.4) Accelerate more advanced functionality like gradients,
composition and antialising -> Accalerate most of Java2Demo, make the
Nimbus LnF work well.
5.5) Extensive profiling an compatibility testing, using self written,
Sun's jtreg and real- world tests, reporting performance and
implementation problems back to driver programmers. Make sure large
graphical applications work well.



6.) Developer
Clemens Eisserer is Student at the Technical University of Vienna
(Information Technology) and:
* Has submitted patches to jdk-collaboration and OpenJDK (Java2D)
* Ported Kaffe's XAWT to GCJ: http://sf.net/projects/xawt
* Is working on an open-source remote-swing project:
http://juibrowser.sourceforge.net/
* Compiled and packaged a free JVM for the Nokia770 internet tablet
(got discounted N800 for that work)
* Finished many commercial projects written in Java with focus on
client-server architectures


7.) Links, References, ...
http://keithp.com/~keithp/render/
http://www.nabble.com/Best-way-organizing-XRender-additions-for-the-X11-pipeline--td14152122.html
http://www.phoronix.com/scan.php?page=article&item=934&num=1