[External] : Re: Proposal: Exposing a GraphicsContext-like API for WritableImage via Software Renderer

[Andy Goryachev]

There is currently a POC PR https://github.com/openjdk/jfx/pull/1836 for headless, so it might be relatively easy to check.

-andy

From: John Hendrikx <john.hendrikx at gmail.com>
Date: Tuesday, July 1, 2025 at 10:36
To: Andy Goryachev <andy.goryachev at oracle.com>, openjfx-dev at openjdk.org <openjfx-dev at openjdk.org>
Subject: [External] : Re: Proposal: Exposing a GraphicsContext-like API for WritableImage via Software Renderer

If you enable the software pipeline, that would not change anything, apart from the libraries already being loaded and the software renderer being used more often. The renderer used in the example would be independent and so it would not interfere.

As far as the headless platform goes, if it allows for the software pipeline to be loaded, then this code should function as well as the Pisces renderer used here is pure Java.  A mock Screen may need to be provided.  I'm not well-versed currently in the specifics of the headless platform.  Perhaps somebody working on it could comment?

--John


On 01/07/2025 18:40, Andy Goryachev wrote:
The devil is in detail, but I like this proposal very much, especially if most of the machinery is already there.

How would it work if one does enable the software pipeline?  Or the headless platform?

-andy


From: openjfx-dev <openjfx-dev-retn at openjdk.org><mailto:openjfx-dev-retn at openjdk.org> on behalf of John Hendrikx <john.hendrikx at gmail.com><mailto:john.hendrikx at gmail.com>
Date: Friday, June 27, 2025 at 03:15
To: openjfx-dev at openjdk.org<mailto:openjfx-dev at openjdk.org> <openjfx-dev at openjdk.org><mailto:openjfx-dev at openjdk.org>
Subject: Proposal: Exposing a GraphicsContext-like API for WritableImage via Software Renderer

Hi list,

I'm exploring whether there is interest in exposing a (Canvas) GraphicsContext-like interface for WritableImage backed by the existing software rendering pipeline.  Currently, JavaFX offers two main choices for drawing and pixel manipulation, each with different trade-offs:

- Canvas provides rich drawing primitives via GraphicsContext, but offers no direct pixel access—requiring costly GPU readbacks (e.g., snapshot()).
- WritableImage, on the other hand, allows direct pixel manipulation via PixelReader/PixelWriter, but has no built-in support for drawing operations like shapes, fills, or blending.

My proposal would combine the strengths of both:

- Expose drawing operations (shapes, fills, etc.) for WritableImage
- Direct access to image data before or after rendering without GPU readbacks / snapshots
- Reuse of JavaFX’s software rendering stack (SWGraphics, PiscesRenderer, etc.) without activating the software pipeline globally

I’ve successfully tested this approach in a non-modular FX application by accessing internal APIs from com.sun.prism.sw. With minor adjustments (--add-exports), it may also work in modular environments.

Work needed to support this as a public API might include:

- Creating a new GraphicsContext-like interface ("DrawingContext" ?)
- Exposing a method on WritableImage to obtain such a context
- Optionally, refactoring Canvas' GraphicsContext to implement this new interface (method signatures are likely compatible)
- Implementing the new interface on top of the software renderer

See the end of the post for a working example (assuming you place the code in the "com.sun.prism.sw" package and deal with the module restrictions).  Note that you do not need to enable the software pipeline (and you don't want to either, as the whole point is to remain GPU accelerated but have software renderer backed drawing primitives for images).

Any feedback appreciated!

--John

package com.sun.prism.sw;

import com.sun.glass.ui.Screen;

import com.sun.glass.utils.NativeLibLoader;

import com.sun.pisces.JavaSurface;

import com.sun.pisces.PiscesRenderer;

import com.sun.pisces.RendererBase;

import com.sun.prism.paint.Color;

import java.nio.IntBuffer;

import javafx.application.Application;

import javafx.scene.Scene;

import javafx.scene.image.ImageView;

import javafx.scene.image.PixelWriter;

import javafx.scene.image.WritableImage;

import javafx.scene.layout.StackPane;

import javafx.stage.Stage;

public class SWRendererExample {

static {

NativeLibLoader.loadLibrary("prism_sw");

}

public static void main(String[] args) {

Application.launch(App.class);

}

public static class App extends Application {

@Override

public void start(Stage primaryStage) {

WritableImage writableImage = createImageWithSWPipeline();

Scene scene = new Scene(new StackPane(new ImageView(writableImage)));

primaryStage.setScene(scene);

primaryStage.show();

}

}

public static WritableImage createImageWithSWPipeline() {

int width = 400;

int height = 300;

SWResourceFactory resourceFactory = new SWResourceFactory(Screen.getMainScreen());

SWRTTexture texture = new SWRTTexture(resourceFactory, width, height);

SWContext swContext = new SWContext(resourceFactory);

// Set up a surface to draw on and create the renderer:

int[] backingArray = new int[width * height];

IntBuffer pixelBuffer = IntBuffer.wrap(backingArray);

JavaSurface surface = new JavaSurface(backingArray, RendererBase.TYPE_INT_ARGB_PRE, width, height);

PiscesRenderer renderer = new PiscesRenderer(surface);

// Create SWGraphics for drawing (software renderer)

SWGraphics swGraphics = new SWGraphics(texture, swContext, renderer);

swGraphics.clear(Color.WHITE);

swGraphics.setPaint(Color.BLUE);

swGraphics.fillRect(50, 50, 100, 100);

swGraphics.setPaint(Color.RED);

swGraphics.fillEllipse(75, 75, 10, 20);

// Take the result and place it in a writable image:

WritableImage writableImage = new WritableImage(width, height);

PixelWriter pw = writableImage.getPixelWriter();

pw.setPixels(0, 0, width, height, javafx.scene.image.PixelFormat.getIntArgbPreInstance(), pixelBuffer.array(), 0, width);

return writableImage;

}

}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/openjfx-dev/attachments/20250701/5ab1ee84/attachment-0001.htm>