Reading binary data

[Samuel Audet]

Hi, Maurizio,

On 11/23/19 8:25 PM, Maurizio Cimadamore wrote:
> Hi Samuel
> 
> On 23/11/2019 05:11, Samuel Audet wrote:
>> Maurizio,
>>
>> To be clear, I'm not advocating for the status quo, far from it. Since 
>> you brought up Jigsaw, let's take as an example. In its current 
>> incarnation, JPMS has been quite literally rammed through over 
>> considerable opposition from the Java community:
>> https://www.theregister.co.uk/2017/05/12/oracle_loses_jpms_vote/
> 
> This is yesterday news. And this pretty summarizes your approach towards 
> this project as a whole: pick up a piece of negative feedback, infer 
> some general consequences from it, rinse and repeat. How much were you 
> involved in the design of the module system? Did you contribute 
> _anything_ on jigsaw-dev (which is where this discussion should happen, 
> BTW) ? I'm guessing not. So, please, leave comments on whether JPMS is 
> good or bad to people who actually have a stake in it.
> 
> What I was actually trying to do was to position some of the things 
> we're exploring for native access in the wider context of what has 
> happened in the Java platform over the last few years, and the general 
> desire of making the Java platform safer, and its configuration more 
> predictable. Whether _you_ like it or not, that's where the platform is 
> going.

It's not a question of whether I like it or not, it's about having the 
tools that we need. Recently, both Ioannis and Ty have been trying to 
make you understand the same thing that I've been trying to make you 
understand about safety, and you're not taking them seriously either. 
How many people is it going to take for you to understand the importance 
of this?

We absolutely need unsafe access to memory regardless of the opinions or 
desires of anyone. Preventing us from having more usable APIs in that 
sense will continue to hamper us in our work, and will only weaken the 
Java platform vs other platforms when it comes to software that depends 
on, for example, accelerators like GPUs. If that's not where OpenJDK 
intends to take the Java platform **in priority**, that's perfectly 
fine, but please make it clear so that we can stop wasting time arguing 
like this.


>> Instead of establishing in priority a weak module system like that or 
>> even developing something like jextract, have you considered that the 
>> Java community at large might have prioritized a "safer JNI"? I don't 
>> see that kind of discussion of priorities happening anywhere at 
>> OpenJDK! You have a fully top-down approach. From what I understand of 
>> our discussions, you do not seem to believe this is a problem, but in 
>> fact it very much /is/ a problem.
>>
> And here we are again, mixing things that have very little to do with 
> each other. What has JPMS have to do with jextract? But, more 
> importantly, why do you keep assuming that nobody cares about jextract? 
> There might be other solutions available for doing what jextract does, 
> but how are you so sure that no Java developer cares about that? The 
> feedback on this very mailing list (from people who actually tried 
> things), as well as some of the comments/questions on social media (most 
> of which were repeatedly shut down by _you_) seem to suggest quite the 
> contrary. How do we establish our priorities? By going to conferences, 
> talking to real developers and customers, taking in feedback, be it 
> positive, or negative. How do _you_ establish what's best for the Java 
> ecosystem as a whole? This is a question that, after a year or so, I'm 
> still struggling to answer.

Well, think about it this way. If OpenJDK prioritized unsafe access to 
memory and low-level APIs similar to JNA, JNR, or Substrate VM's, or 
anything else that is actually usable without a parser like jextract, 
then the **whole** community would be able to work on tools on top of 
those low-level APIs that you provide.

Right now, all those low-level APIs are restricted, internal, not even 
part of Java SE, let alone Android, and they're not even yet faster than 
JNI or sun.misc.Unsafe, and actually slower for AOT compilers. Moreover, 
they are always changing and because of this lack of stability, most of 
the community is not able to use them extensively enough to provide 
(more) feedback. It's not possible to do anything substantial with an 
API that is so unstable! Although you're not willing to talk about what 
you might have to do to support C++, I can imagine that it's going to 
end up making even more aggressive changes, but maybe not? Not knowing 
what your intentions are isn't helping.

If those low-level bits could be stabilized first, then I'm sure that 
the rest of the community (millions of them) would start experimenting 
on them, including myself. That's how you work with the community: You 
give them what they need to help you. On its own by itself, OpenJDK is 
never ever going to be able to provide everything that customers that 
you meet at those conferences require. Compromises need to be made.


>> I admit my attitude isn't ideal, but I'm still trying to make you 
>> understand my point in this. Let's try to work on features that are 
>> most useful to everyone, first. We all have limited resources, so 
>> let's try to make the most of them, and that means working with the 
>> community! In any case, I'll keep trying to do my best and be as least 
>> annoying as possible. :)
> 
> The real elephant in the room here is that we've been arguing for over 
> an year _without you having written a single line of code_ using 
> whatever we're doing. In reality, I don't see why there couldn't be an 
> experimental version of JavaCPP targeting memory access VH and MH 
> instead of JNI - then it would be good to have a discussion about what 
> worked and what didn't. I think until you do something along those 
> lines, not many people will really take you seriously around here, sorry.

We've talked about this before. Once you're able to provide a viable 
alternative to JNI, that runs faster for both JIT and AOT compilers, 
whose API is somewhat stable, that allows full unsafe access somehow, 
and that is slated to become part of Java SE, even just as an initial 
draft of a JEP, then I will start to look at it.

For what it's worth, I think the problem is in part that jextract has 
been prioritized, but this tool has no chance whatsoever of making it 
into Java SE. Doing so would require writing specs for essentially the 
whole of C/C++ and LLVM. That's never going to happen... unless OpenJDK 
decides to do without the JCP and JSRs entirely, at which point we're 
going to have everyone creating forks of the JDK. Either way, something 
will happen, and if we have to wait until all hell breaks loose like 
this, I've made my peace with that. I did what I could to prevent it. 
Issues will get resolved one way or another. Before we get to that point 
though, if Panama decides to cooperate more with the community and 
accepts that compromises need to be made, I will be here to help!

Samuel