AW: Foreign memory access with classes

Wed Nov 9 14:17:48 UTC 2022

Prototype for native classes

//example C-Struct Point:

struct Point {

    int x,y;

};

//the base class Native:

import java.lang.foreign.MemorySegment;

import java.util.Objects;

public abstract class Native {

    protected final MemorySegment segment;

    protected Native(MemorySegment segment) {

        this.segment = Objects.requireNonNull(segment);

        if (segment.byteSize() != layout().byteSize())

            throw new IllegalArgumentException("Segment does not match native class.");

    }

    public MemorySegment segment() {

        return segment;

    }

    public abstract GroupLayout layout();

}

//the declaration of the native class Point:

public native class Point(public int x, public int y) {}

//the expansion of the syntax above:

public final class Point extends Native {

    private static final GroupLayout $LAYOUT = MemoryLayout.structLayout(

        ValueLayout.JAVA_INT.withBitAlignment(32).withName("x"),

        ValueLayout.JAVA_INT.withBitAlignment(32).withName("y"));

    private static final VarHandle X$HANDLE = $LAYOUT.varHandle(MemoryLayout.PathElement.groupElement("x"));

    private static final VarHandle Y$HANDLE = $LAYOUT.varHandle(MemoryLayout.PathElement.groupElement("y"));

    public Point(MemorySegment segment) {

        super(segment);

    }

    public int getX() {

        return (int)X$HANDLE.get(segment);

    }

    public void setX(int x) {

        X$HANDLE.set(segment, x);

    }

    public int getY() {

        return (int)Y$HANDLE.get(segment);

    }

    public void setY(int y) {

        Y$HANDLE.set(segment, y);

    }

    @Override

    public GroupLayout layout() {

        return $LAYOUT;

    }

}

The expansion would be done by the compiler at compile time just like records.

As said this is just a prototype and different or additional ideas are very welcome.

The visibility of the accessor methods can be set in the field declaration brackets.

To change visibility of only getter or setter you would simply choose the lower visibility and define a custom one instead of the default.

To access the default in the custom method super could be used.

As you might have notices this looks similar to the class jextract creates and that’s intentional.

If this would be implemented jextract could just create native classes as bindings to native structs.

This would practically shift the modeling of the concrete c struct from jextract to the javac compiler.

Since the native struct declaration is very simplistic the need for jextract could be reduced.

I think the ability to describe a native struct in 1 line instead of using an extra tool would simplify the process of interfacing with native structs a lot. 

The bindings created by jextract can look confusing and scary to many less experienced java developers.

Working with a native class is the exact same as working with a mutable record like class with the benefits that it can be used to interact with native memory.

Thinking further jextract could actually insert the generated native classes in signatures of the java representation of native functions.

This idea would promote native structs to first class citizens in java.

In great regards

RedIODev

Von: Red IO <redio.development at gmail.com> 
Gesendet: Dienstag, 8. November 2022 14:01
An: Maurizio Cimadamore <maurizio.cimadamore at oracle.com>
Cc: panama-dev <panama-dev at openjdk.org>
Betreff: Re: Foreign memory access with classes

JPassport looks like a great take based on the raw Panama way of doing this. But it's still a pain to do and requires a precompiler. But both feel uncomfortable using strings to point to mark things.

Also my idea would still bahave like a normal class being on the heap (could be declared as primitive or value class but let's not focus on Valhalla). The only difference would be their fields, as they would technically not be part of the class. The only field part of the object would be the hidden MemorySegment managing the access of the fields.

My biggest problem with the current solution is that they try to simulate a simple struct with a complex factory. I understand your concerns but there is no split in the type system like you feared. Native objects are simply different, that their fields are stored somewhere else. Other than that they are plain objects. If projecting method calls from fields is too complicated to implement we could go for a record like syntax and only exposing the fake fields through methods. Most java developers would probably never create their own native classes but use those declared in libraries and returned by library methods. In this scenario they would be indistinguishable from normal objects.

Regards 

RedIODev

On Tue, Nov 8, 2022, 11:44 Maurizio Cimadamore <maurizio.cimadamore at oracle.com <mailto:maurizio.cimadamore at oracle.com> > wrote:

I believe adding Java syntax to model objects backed by memory segments 
might be pushing things a little too far. From a pedagogical perspective 
you would now have to explain to _every_ Java developers that there are 
two different kinds of objects, plain and native, and the rules which 
govern their access would be different (plain objects are 
garbage-collected, native objects are not).

Seems like a nightmare, for a relatively little pay off. Note that, if 
you declare a record class with components X and Y, it is relatively 
easy to construct a function that can read a segment, with given struct 
layout into a record with matching components. I believe JPassport [1] 
does something in this direction. With something like this you get 
basically 90% of what you are aiming for, without the need to change the 
language by adding a new keyword, and making the programming model more 
complex for all the developers out there, including those who do not 
care about off-heap access.

Cheers
Maurizio

[1] - https://github.com/boulder-on/JPassport/

On 08/11/2022 06:52, Red IO wrote:
> I just had a crazy idea on how to make foreign memory access work and 
> feel like java native class member access.
> The idea is that you define special classes maybe with an interface or 
> a special class keyword. This class can then be "constructed" using a 
> MemorySegment being bound by its special and temporal bounds. The 
> object would be either null or every memory access throws an exception 
> when the bounds are breached. The class would be implicitly final and 
> would extend a class Native.
> This could look something like this :
>
> //c struct
> typedef struct {
> int x;
> int y;
> } Point;
>
> //java native class
> public native class Point {
>
> int x;
> int y;
>
> //implicit and forced private constructor
> //normal creation would make things difficult
> }
>
> MemorySegment data =... //native memory
> Point p = data.object(Point.class);
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20221109/41661c97/attachment-0001.htm>