Proposal: Exposing a GraphicsContext-like API for WritableImage via Software Renderer

[John Hendrikx]

If you enable the software pipeline, that would not change anything,
apart from the libraries already being loaded and the software renderer
being used more often. The renderer used in the example would be
independent and so it would not interfere.

As far as the headless platform goes, if it allows for the software
pipeline to be loaded, then this code should function as well as the
Pisces renderer used here is pure Java.  A mock Screen may need to be
provided.  I'm not well-versed currently in the specifics of the
headless platform.  Perhaps somebody working on it could comment?

--John


On 01/07/2025 18:40, Andy Goryachev wrote:
>
> The devil is in detail, but I like this proposal very much, especially
> if most of the machinery is already there. 
>
>  
>
> How would it work if one does enable the software pipeline?  Or the
> headless platform?
>
>  
>
> -andy
>
>  
>
>  
>
> *From: *openjfx-dev <openjfx-dev-retn at openjdk.org> on behalf of John
> Hendrikx <john.hendrikx at gmail.com>
> *Date: *Friday, June 27, 2025 at 03:15
> *To: *openjfx-dev at openjdk.org <openjfx-dev at openjdk.org>
> *Subject: *Proposal: Exposing a GraphicsContext-like API for
> WritableImage via Software Renderer
>
> Hi list,
>
> I'm exploring whether there is interest in exposing a (Canvas)
> GraphicsContext-like interface for WritableImage backed by the
> existing software rendering pipeline.  Currently, JavaFX offers two
> main choices for drawing and pixel manipulation, each with different
> trade-offs:
>
> - Canvas provides rich drawing primitives via GraphicsContext, but
> offers no direct pixel access—requiring costly GPU readbacks (e.g.,
> snapshot()).
> - WritableImage, on the other hand, allows direct pixel manipulation
> via PixelReader/PixelWriter, but has no built-in support for drawing
> operations like shapes, fills, or blending.
>
> My proposal would combine the strengths of both:
>
> - Expose drawing operations (shapes, fills, etc.) for WritableImage 
> - Direct access to image data before or after rendering without GPU
> readbacks / snapshots
> - Reuse of JavaFX’s software rendering stack (SWGraphics,
> PiscesRenderer, etc.) without activating the software pipeline globally
>
> I’ve successfully tested this approach in a non-modular FX application
> by accessing internal APIs from com.sun.prism.sw. With minor
> adjustments (--add-exports), it may also work in modular environments.
>
> Work needed to support this as a public API might include:
>
> - Creating a new GraphicsContext-like interface ("DrawingContext" ?)
> - Exposing a method on WritableImage to obtain such a context
> - Optionally, refactoring Canvas' GraphicsContext to implement this
> new interface (method signatures are likely compatible)
> - Implementing the new interface on top of the software renderer
>
> See the end of the post for a working example (assuming you place the
> code in the "com.sun.prism.sw" package and deal with the module
> restrictions).  Note that you do not need to enable the software
> pipeline (and you don't want to either, as the whole point is to
> remain GPU accelerated but have software renderer backed drawing
> primitives for images).
>
> Any feedback appreciated!
>
> --John
>
> *package*com.sun.prism.sw;
>
> *import*com.sun.glass.ui.Screen;
>
> *import*com.sun.glass.utils.NativeLibLoader;
>
> *import*com.sun.pisces.JavaSurface;
>
> *import*com.sun.pisces.PiscesRenderer;
>
> *import*com.sun.pisces.RendererBase;
>
> *import*com.sun.prism.paint.Color;
>
> *import*java.nio.IntBuffer;
>
> *import*javafx.application.Application;
>
> *import*javafx.scene.Scene;
>
> *import*javafx.scene.image.ImageView;
>
> *import*javafx.scene.image.PixelWriter;
>
> *import*javafx.scene.image.WritableImage;
>
> *import*javafx.scene.layout.StackPane;
>
> *import*javafx.stage.Stage;
>
> *public**class*SWRendererExample {
>
> *static*{
>
> NativeLibLoader./loadLibrary/("prism_sw");
>
> }
>
> *public**static**void*main(String[] args) {
>
> Application./launch/(App.*class*);
>
> }
>
> *public**static**class*App *extends*Application {
>
> @Override
>
> *public**void*start(Stage primaryStage) {
>
> WritableImage writableImage = /createImageWithSWPipeline/();
>
> Scene scene = *new*Scene(*new*StackPane(*new*ImageView(writableImage)));
>
> primaryStage.setScene(scene);
>
> primaryStage.show();
>
> }
>
> }
>
> *public**static*WritableImage createImageWithSWPipeline() {
>
> *int*width = 400;
>
> *int*height = 300;
>
> SWResourceFactory resourceFactory =
> *new*SWResourceFactory(Screen./getMainScreen/());
>
> SWRTTexture texture = *new*SWRTTexture(resourceFactory, width, height);
>
> SWContext swContext = *new*SWContext(resourceFactory);
>
> // Set up a surface to draw on and create the renderer:
>
> *int*[] backingArray = *new**int*[width * height];
>
> IntBuffer pixelBuffer = IntBuffer./wrap/(backingArray);
>
> JavaSurface surface = *new*JavaSurface(backingArray,
> RendererBase.TYPE_INT_ARGB_PRE, width, height);
>
> PiscesRenderer renderer = *new*PiscesRenderer(surface);
>
> // Create SWGraphics for drawing (software renderer)
>
> SWGraphics swGraphics = *new*SWGraphics(texture, swContext, renderer);
>
> swGraphics.clear(Color.WHITE);
>
> swGraphics.setPaint(Color.BLUE);
>
> swGraphics.fillRect(50, 50, 100, 100);
>
> swGraphics.setPaint(Color.RED);
>
> swGraphics.fillEllipse(75, 75, 10, 20);
>
> // Take the result and place it in a writable image:
>
> WritableImage writableImage = *new*WritableImage(width, height);
>
> PixelWriter pw = writableImage.getPixelWriter();
>
> pw.setPixels(0, 0, width, height,
> javafx.scene.image.PixelFormat./getIntArgbPreInstance/(),
> pixelBuffer.array(), 0, width);
>
> *return*writableImage;
>
> }
>
> }
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/openjfx-dev/attachments/20250701/ced6ba53/attachment-0001.htm>