Panama feedback

[Maurizio Cimadamore]

On 30/03/2020 21:35, Ty Young wrote:
> Hi all,
>
>
> After doing abstraction layer, multiple hand-made bindings, and an 
> entire API swap from old Pointer API to new foreign memory-access at 
> the OO layer, my JavaFX application has been rebased on current 
> Panama. There are a few things that have come out of it.
>
>
> For reference, the old project dependency hierarchy looked like:
>
>
> Pointer Bindings -> OO API -> JavaFX
>
>
> The new dependency hierarchy looks like:
>
>
> Foreign Memory Access API -> abstraction layer -> hand-made bindings 
> -> OO API -> JavaFX
>
>
> The JavaFX application has a single background thread that updates the 
> GPUs attribute information(like 50 of them) with a 1 second wait 
> between updates. I'm also using a newer version(14) of JavaFX right 
> now then I was then with the Pointer API.
Thanks for the perseverance and for trying out the new stuff!
>
>
> The first being that *there isn't, AFAIK, a way to find *native* 
> memory leaks.* Most object references within the OO API are final, as 
> there will always need to be, for example, an int pointer to read a 
> GPUs temp for the life time of that GPU.
>
> However, hypothetically if there was a memory leak, I don't know how 
> i'd go about finding it. A few bytes memory leak is hardly noticeable 
> even after hours of use. If there isn't already, It would be nice if 
> there was a way to profile native memory allocations via MemoryAccess.

I hear you - one of the reasons we might want an allocator other than 
malloc, beside performance, is precisely this: diagnosability 
(especially if such allocator would be used JDK-wide, even by native 
code). This of course doesn't fully protect from native memory allocated 
by the API, but it would cover all the one allocated via the memory 
segment API.

Then there's the problem of forgetting to close() some segments; and I 
think Netty did some experiments in this area to run the code in a 
special, slower mode which reported error for every buffer which became 
unreacheable before its close method could be called. Something along 
those lines could be added.

>
>
> Going along with that, *there seems to be a significant increase in 
> allocations.* I don't have a build of the old Pointer JDK so I can't 
> compare, but at about 135MB min heap sizes according to Netbeans's 
> profiler, it feels like there is a lot of needless allocations happening.
>
> It might just be JavaFX(which is known to have memory issues), I don't 
> know, but according to Netbean's profiler *something* is allocating MB 
> worth of HashMaps and HashNodes and heap allocations of those goes as 
> high as 30 MB before dropping to 8 MB once a GC is finished. There 
> doesn't seem to be any leaks per se, just a crazy amounts of 
> allocations that are then thrown away. Without better memory profiler 
> tools(jProfiler) I can't really see what's causing it.
>
While the native interop API is not in a state where we can say that 
we've optimized it for memory footprint, what you say sounds weird. The 
only widespread-enough use of maps that I can think of, is that for 
storing memory layouts attributes. So, when creating lots and lots of 
garbage layouts, I guess it's also possible to create lots and lots of 
garbage maps.

As for profilers, in the past I found Eclipse Memory Analyzer quite 
useful and fast in processing even gigabytes of heap. Perhaps you could 
take a look at that.

>
> Moving away from memory issues, *I'd like to see a feature added to 
> jextract to be able to just dump layout information*. I have no 
> interest in using jextract for creating bindings as I have said 
> before, but a tool for generating the correct struct layouts is still 
> very useful. I'd rather it just dump the information to console rather 
> than to on-disk too, if it's possible.
>
>
> *As a bonus, it would be nice if you didn't need to include every 
> required header file that would normally be required to generate 
> jextract bindings.***In other words, if a struct uses a type which is 
> defined in another header that isn't included, a warning is thrown and 
> it just spits out layouts for what it can figure out.

These are all good suggestions and part of a bigger topic that at some 
point we'll need to revisit about how to filter bindings in the optimal 
way. Generating only layouts is another way to filter the binding output 
to only include types (for which we're thinking about adding a lot more 
helper functions than we do at the moment, we're doing some experiments 
there).

There are many ways to skin the cat there:

* command line options
* user-defined plugins/visitors
* side-files which contain list of desired symbols to be extracted

As you say with filtering there's always a problem of defining what 
happens when you have a symbol S1 which is inside the filter which 
refers to a symbol S2 which is outside the filter; again, many possible 
behaviors:

* error - stop
* warning - try to figure out something that still makes some kind of sense
* force-include what was left out

While I don't have a specific solution to propose at this time, my 
feeling is that we should provide tools to help people to generate as 
small bindings as possible; and then perhaps have a 'I don't care mode' 
where the tool is allowed to pull in the entire world. But the former 
mode is gonna be crucial in context where you need to limit the amount 
of static footprint and classes being loaded.


>
>
> As for major API gripes,*l**ooking at the new "Foreign" API, it feels 
> to me like it's a bit too thin and there isn't enough there to justify 
> it.* Taking out the string utility code leaves 3 methods, one of which 
> is just a getter for SystemABI.
This is something for which we still have to find the right balance. The 
string utility might go soon, as part of the experiment I mentioned 
above - we don't exclude that, at some point, we might be able to take 
out Foreign and scatter unsafe API points around (with adequate 
disclaimers in the javadoc). This is something that will likely be 
decided near the end of the process when we know exactly how many unsafe 
interop API points we need. So let's not focus too much on what it looks 
like _now_.
>
>
> Personally I'd like to see SystemABI reverted back and provide both 
> platform specific and runtime layouts. An interface which defines a 
> standard way of getting standard cross-platform types and the ABI name 
> can be defined and implemented by "WindowsABI", "SysVABI", and 
> "AArch64ABI" classes. SystemABI would then provide those classes via 
> static constants(not the interface implementations, the actual class 
> since platforms have their own specific layout types) and implement 
> the interface so that it provides current ABI layouts.
Right now we're trying hard to leave SystemABI language agnostic (in 
fact, current SystemABI should be renamed C_ABI, or something like that) 
- the reasoning is that now, the underlying machinery is powerful enough 
that we can add more ABIs in the future as we see fit.
>
>
> (Or just have a getABI() method. Probably the easiest.)
>
>
> I think at that point what remains of MemoryLayouts(Java, padding, 
> etc) could then be merged into MemoryLayout. This would add a better 
> divide, making things less blurry, I think. Runtime layouts are 
> currently just thrown into MemoryLayouts right next to Java ones and 
> it feels a bit off.
That's the plan, yes, and was also mentioned somewhere in the JEP 370 RFR.
>
>
> A minor API gripes is that, *currently, there is no way to create zero 
> length arrays.* I think this can be added by introducing "INVALID" 
> constant SequenceLayout which, when passed to AllocateNative, results 
> in AllocateNative returning MemoryAddress.NULL. Since the byte size of 
> MemoryAddress.NULL is zero, I feel like this should be a safe thing to 
> do.

Uhm - not sure; why would you want to create the layout of a zero-length 
array? To support VLA? E.g.

struct Foo {
    int nfoos;
    int foos[0];
}

If so, I'm not sure having a zero-length layout is going to help you 
much - you still have to tell the API how much memory to allocate for 
that struct, which is, essentially, a dynamic thing.

When it comes to accessing the above struct, you can use sequences 
without a size:

MemoryLayout.ofStruct(
     MemoryLayouts.JAVA_INT.withName("nfoos"),
MemoryLayout.ofSequence(MemoryLayouts.JAVA_INT)).withName("foos"))

And this will allow you to dereference elements as you do in C (with the 
appropriate VarHandle). The only thing you can't do with this layout is 
to use it for allocation - but a zero-length sequence won't help much 
either there.


>
>
> *There is also currently no explicit null input argument checks for 
> things like unsafe operations methods.* Don't know if that's 
> intentional or not, but if possible, I'd like to see explicit checks, 
> especially on "unsafe" operations. I had a bit of a "WTF" moment when 
> making stdlib bindings because I forgot to change 
> FunctionDescriptor/MethodHandle to return something which resulted in 
> a semi-unhelpful NullPointerException when I attempted to make a null 
> unsafe MemoryAddress safe.
That's a bug - we should fix it. Unsafe operations are not meant to be 
that unpolished - but we have to work on that. Again, once we will have 
a better sense of exactly which set of operations we need to add, we'll 
make them more polished too, and they will check and throw all the right 
exceptions, which will be documented in the javadoc. Think of that as 
work in progress, for now.
>
>
> (That new JDK 14 Helpful NullPointerExceptions feature is really nice, 
> BTW!)
Yep - I like that too!
>
>
> Maybe i'm just blind but *there doesn't seem to be a way to do 
> callbacks*. I'm guessing since you use downcall to get a function 
> address that you do upcall, but I'm not sure how to go about using it.
You mean? Passing Java code as a function pointer to some native call? 
Sure there is a way to do that! If this is really what you are looking 
for I can give you pointer to some examples using that feature.
>
>
> That pretty much covers it. I can't speak about performance or 
> anything else since what I'm using Panama for doesn't require every 
> last drop of performance. I can say that weird oddities like getting 
> random numbers from a Pointer<Integer>, for example, no longer happen 
> which is nice. Getting a really large, seemingly random number when 
> asking what the GPU's utilization(0-100%) was was a bit "WTF".
>
>
> I should also probably take the time here to say thanks to everyone 
> who's working on this. I really appreciate the amount of time, 
> thought, and overall effort that is being put into this. Panama is 
> making native binding creation a breeze so far. Many thanks to everyone!

Thanks, this is very good feedback; not surprisingly most of the points 
you raise are related to aspects for which we know we have to do better 
- and we have some stuff in the pipeline which is meant to address some 
of these issues; I'd say at this point, we have to keep iterating until 
it "feels right", and this is something that only real experiments (like 
the one you did) can cover in full. As a side note, Sundar has been 
working like crazy lately to port most of the old panama/foreign 
examples to the new API and we're learning quite a bit from that 
exercise. It feels like we have all the bits we need, but of course it 
might take some time until we arrange the pieces of the puzzle in the 
lowest energy configuration ;-)

One question for you - given some feedback you gave recently on some of 
the messages I posted on a pull request; after having tried this stuff 
some more, what is your feeling towards the choice of making native 
pointers non-de-referenceable by default? You seemed to be very 
skeptical at first on this one - but your message the other day gave me 
the impression that you might have a different opinion now; is my 
reading correct? E.g. is it something that, after getting used to, gets 
better, or that you don't bump into it too often, or is it just plain 
annoying - based your experiments?

Cheers
Maurizio

>
>