some thoughts on panama/jextract

[Maurizio Cimadamore]

While I agree with the general spirit of your comments, I think your 
email is missing a central point here: this is a Java API. The potential 
for an API such as this to be abused by developers that are not as 
versed in C as you are are immense. So, I'd say that there is another 
angle for enforcing the restriction we are talking about which is (3) - 
pedagogical: make developers stop and think before using a pointer that 
comes from a native library which can feature spatial and temporal 
bounds which are simply not known statically.

Note that, while it's true that with memcpy you can get around things, 
you still need to get a native MethodHandle to be able to call memcpy in 
the first place, and obtaining that handle will be a restricted 
operation in the future (and it has to be, since the runtime has to 
trust the developer on what native signature does a native function have 
- given that signature is not captured in the .so or .dll file).

The situation right now is suboptimal, because the API we have now seems 
to suggest that creating native method handles is fair game, whereas 
changing pointer size is not. In reality both will be restricted 
operation which will require some kind of flags to be passed on the 
command line. I understand this might not be what you were looking for, 
or that you might find this restriction too hand-holding for your 
tastes, but I think that's a necessary price to pay when you are adding 
features like these (which are application breaking, if misused) to a 
runtime which is used by so many developers.

Maurizio


On 13/01/2020 00:53, Michael Zucchi wrote:
> On 10/1/20 9:10 pm, Maurizio Cimadamore wrote:
>>
>> The point I'm making is about what the default should be. I think 
>> there's no question that, at the expressiveness level, you can make 
>> things to work in the current scheme. The only real question here is 
>> one: should pointers returned by native libraries be de-referenceable 
>> by default or not? 
>
> I don't see how that's the only real question, and honestly I don't 
> see how it's even a question in the first place.  C support simply 
> requires both and which is used in any case is *explicitly defined by 
> the API you're accessing*. Thus in the context of "calling C 
> functions", having one way or another as a "default" doesn't have any 
> meaning because there is simply no such default in C.  C relies on 
> both being available to be able to efficiently do useful work and 
> there are technical trade-offs when using either approach.
>
> Therefore any 'default' is not technical but political and the purpose 
> seems to be to:
>
>  1. Inform the binding-developer they have to be extra careful and
>     that they're using a 'bad' library (according to some specific but
>     ultimately arbitrary definition of 'bad'); and
>  2. Inform the binding-user-developer they are using something
>     potentially dangerous or 'unsafe'.
>
> But 1 is mostly meaningless and anyway redundant - and frankly 
> insulting, and 2 is actively misleading as i demonstrated with the 
> memcpy example.
>
> And you keep using loaded language like 'ill behaved' for one of the 
> cases.  But this title is simply not true at face value in a technical 
> sense and comes across as politically charged passive-aggressive 
> framing.  Such language is neither helpful nor necessary.
>
> I have less time to look at this now i'm back at work but will 
> continue my experiments* and report on them, and will follow the 
> project to see what you guys come up with.  From past experience i'm 
> pretty confident you'll come up with a decent solution in the end.  
> But it's interesting to see the process in motion and perhaps 
> contribute to its shaping rather than just passively consume the 
> results as I typically do.
>
> Regards,
>  Michael
>
> [*] This is primarily a deep deep 30+ year hobby for me that i've 
> basically dedicated/wasted my life to/on.  While I use most things I 
> learn for work and I learnt a great deal from my hobby in order to do 
> that better, my personal driver is honing the craft of turning a cpu 
> into an efficient and useful tool.  I have a lot of free time that I 
> often spend on all sorts of obtuse details, it's my main source of 
> entertainment and its typically about the journey and not the destination.
>
> But debating is definitely not one of the things I enjoy spending my 
> time on!
>