Reading binary data

[Ty Young]

On 12/4/19 10:55 AM, Maurizio Cimadamore wrote:
>
> On 04/12/2019 16:14, Samuel Audet wrote:
>> Hi, Maurizio,
>>
>> It's not a question of whether I like it or not, it's about having 
>> the tools that we need. Recently, both Ioannis and Ty have been 
>> trying to make you understand the same thing that I've been trying to 
>> make you understand about safety, and you're not taking them 
>> seriously either. How many people is it going to take for you to 
>> understand the importance of this?
> I think you are just mixing (as usual) pears with apples. But I will 
> let Ty and Ioannis to speak for themselves, rather than try to put 
> words into their mouths.


Ioannis and I's view on the current "safety" mentality seems to be the 
same: it just doesn't matter from a C/C++ bindings perspective. You say 
the point of "safety" is to not crash the JVM, but the JVM runs on the 
system and C/C++ is often used for sensitive system level programming. 
Even if you did everything possible to ensure the integrity of the way 
the data is communicated and worked with between Java and C/C++, it 
literally does nothing once that data is processed in C/C++ land. You 
crash the system from C/C++ and the JVM will go down with it.


That said, I don't think "safety" is pointless and I recognize that 
there are many use cases where "safety" is very much welcomed. However, 
by not offering functionality because of "safety", you are shafting 
those that want to get "down and dirty" using Project Panama... 
something Java doesn't really have an official API for.


>>
>> We absolutely need unsafe access to memory regardless of the opinions 
>> or desires of anyone. Preventing us from having more usable APIs in 
>> that sense will continue to hamper us in our work, and will only 
>> weaken the Java platform vs other platforms when it comes to software 
>> that depends on, for example, accelerators like GPUs. If that's not 
>> where OpenJDK intends to take the Java platform **in priority**, 
>> that's perfectly fine, but please make it clear so that we can stop 
>> wasting time arguing like this.
>
> Who is *We*, *us* ? As for wasting time arguing - I'd very much like 
> to stop (as you do). So - maybe stop arguing?
>
> You seem to be determined in having me admitting that some of the 
> important use cases *you* (again, don't know who *you* is) have in 
> mind is not on our radar - and you infer that from the fact that we'd 
> like to have a clear separation between safe parts of the API and less 
> safe parts of the API. That is simply an incorrect inference, which is 
> more similar to FUD than to a real argument.
>

IIRC some parts are still "unsafe" though, are they not?


>
>
>> and actually slower for AOT compilers. Moreover, they are always 
>> changing and because of this lack of stability, most of the community 
>> is not able to use them extensively enough to provide (more) 
>> feedback. It's not possible to do anything substantial with an API 
>> that is so unstable! Although you're not willing to talk about what 
>> you might have to do to support C++, I can imagine that it's going to 
>> end up making even more aggressive changes, but maybe not? Not 
>> knowing what your intentions are isn't helping.
>
> I think these is a big ball of excuses as to why _you_ are not doing 
> anything with the API. Luckily, in this very mailing list, I've seen 
> plenty of people who, unlike you, were willing to get their hands dirty.
>
> As for stability, the memory access API did not have any significant 
> API change in months - and now it's a JEP. I'm also not aware of any 
> breaking changes in the foreign API which is being distributed through 
> EA. There's not a single piece of concrete evidence in anything you 
> say. This fact alone is a big red herring.


In fairness, if one was to read this mailing list, it does sound like 
the API is still being hammered out. It's important to keep in mind that 
the people participating in the mailing lists aren't always reading the 
code or changesets and it isn't like there is a monthly progress report 
on these Java projects either. That's why those conferences that you 
OpenJDK/Oracle developers present at are so important - it lets people 
know what the current status, plans, and thought process are for things 
being worked on in Java.


>>
>>
>>>> I admit my attitude isn't ideal, but I'm still trying to make you 
>>>> understand my point in this. Let's try to work on features that are 
>>>> most useful to everyone, first. We all have limited resources, so 
>>>> let's try to make the most of them, and that means working with the 
>>>> community! In any case, I'll keep trying to do my best and be as 
>>>> least annoying as possible. :)
>>>
>>> The real elephant in the room here is that we've been arguing for 
>>> over an year _without you having written a single line of code_ 
>>> using whatever we're doing. In reality, I don't see why there 
>>> couldn't be an experimental version of JavaCPP targeting memory 
>>> access VH and MH instead of JNI - then it would be good to have a 
>>> discussion about what worked and what didn't. I think until you do 
>>> something along those lines, not many people will really take you 
>>> seriously around here, sorry.
>>
>> We've talked about this before. Once you're able to provide a viable 
>> alternative to JNI, that runs faster for both JIT and AOT compilers, 
>> whose API is somewhat stable, that allows full unsafe access somehow, 
>> and that is slated to become part of Java SE, even just as an initial 
>> draft of a JEP, then I will start to look at it.
>
> Well, (I start to sound like a broken record) the memory access API is 
> a JEP now; any plans to replace JavaCPP JNI-based struct access with 
> that? Surely that should be much faster?
>
> But in general your attitude of "I will touch it only when it's fully 
> done" makes me think that you are essentially looking at the wrong 
> mailing list. This is for people who likes to makes their hands dirty 
> and play a bit with what's available - if nobody did that (and 
> followed your approach), the feedback we'd get would be exactly zero, 
> nada, nil - would you care to formulate how such an attitude is going 
> to contribute to make whatever we are doing _better_ ?


It's kind of understandable that someone might not spend time converting 
when the thing they are potentially converting to doesn't appear to be 
hammered out yet.


...and Project Panama is made up of various pieces that can't be easily 
assembled yet. Again, for one of the things I'm using Project Panama for 
personally, I'm using jextract primarily and eventually memory-access to 
fill in the missing parts. If there was a branch that included 
everything then people could checkout the various parts of Project 
Panama more easily and in more realistic situations.


Like, is everyone *really* going to use memory access to create bindings 
when they have the complete headers and can use jextract?


Likewise, building the JDK from source takes a lot of processing power 
to do. About 7 minutes on my Ryzen 1800x, in fact. Github Actions could 
probably build the JDK every commit if someone was to set it up. Don't 
know how long it'd take...


>> Samuel