Reading binary data

Sat Nov 23 11:25:03 UTC 2019

Hi Samuel

On 23/11/2019 05:11, Samuel Audet wrote:
> Maurizio,
>
> To be clear, I'm not advocating for the status quo, far from it. Since 
> you brought up Jigsaw, let's take as an example. In its current 
> incarnation, JPMS has been quite literally rammed through over 
> considerable opposition from the Java community:
> https://www.theregister.co.uk/2017/05/12/oracle_loses_jpms_vote/

This is yesterday news. And this pretty summarizes your approach towards 
this project as a whole: pick up a piece of negative feedback, infer 
some general consequences from it, rinse and repeat. How much were you 
involved in the design of the module system? Did you contribute 
_anything_ on jigsaw-dev (which is where this discussion should happen, 
BTW) ? I'm guessing not. So, please, leave comments on whether JPMS is 
good or bad to people who actually have a stake in it.

What I was actually trying to do was to position some of the things 
we're exploring for native access in the wider context of what has 
happened in the Java platform over the last few years, and the general 
desire of making the Java platform safer, and its configuration more 
predictable. Whether _you_ like it or not, that's where the platform is 
going.

>
> Instead of establishing in priority a weak module system like that or 
> even developing something like jextract, have you considered that the 
> Java community at large might have prioritized a "safer JNI"? I don't 
> see that kind of discussion of priorities happening anywhere at 
> OpenJDK! You have a fully top-down approach. From what I understand of 
> our discussions, you do not seem to believe this is a problem, but in 
> fact it very much /is/ a problem.
>
And here we are again, mixing things that have very little to do with 
each other. What has JPMS have to do with jextract? But, more 
importantly, why do you keep assuming that nobody cares about jextract? 
There might be other solutions available for doing what jextract does, 
but how are you so sure that no Java developer cares about that? The 
feedback on this very mailing list (from people who actually tried 
things), as well as some of the comments/questions on social media (most 
of which were repeatedly shut down by _you_) seem to suggest quite the 
contrary. How do we establish our priorities? By going to conferences, 
talking to real developers and customers, taking in feedback, be it 
positive, or negative. How do _you_ establish what's best for the Java 
ecosystem as a whole? This is a question that, after a year or so, I'm 
still struggling to answer.
> I admit my attitude isn't ideal, but I'm still trying to make you 
> understand my point in this. Let's try to work on features that are 
> most useful to everyone, first. We all have limited resources, so 
> let's try to make the most of them, and that means working with the 
> community! In any case, I'll keep trying to do my best and be as least 
> annoying as possible. :)

The real elephant in the room here is that we've been arguing for over 
an year _without you having written a single line of code_ using 
whatever we're doing. In reality, I don't see why there couldn't be an 
experimental version of JavaCPP targeting memory access VH and MH 
instead of JNI - then it would be good to have a discussion about what 
worked and what didn't. I think until you do something along those 
lines, not many people will really take you seriously around here, sorry.

Maurizio

>
> Samuel
>
>
> On 11/18/19 8:33 PM, Maurizio Cimadamore wrote:
>>
>> On 18/11/2019 01:26, Samuel Audet wrote:
>>> Anyway, you're saying you can't make more information public at this 
>>> point about that, that's too bad. As with anything related to C++, 
>>> I'm simply trying to get as much information as I can. OpenJDK 
>>> should really be more "open"... 
>>
>> I think you are mistaking the fact that we're busy spinning many 
>> plates with a desire to keep information less public that desirable. 
>> Right now we're focused on what's ahead of us - which is the memory 
>> access API. Since there's no ABI support slated for this first 
>> deliverable, all these issues around forging unsafe pointers are, 
>> frankly, out of scope at the moment. The goal is to provide a safe 
>> and more efficient alternative to the off heap uses of the ByteBuffer 
>> API.
>>
>> We know that, moving forward to the second part of the story (ABI 
>> integration) we will need eventually to do something to sort out 
>> access to unsafe API. And, moving even further, we know we will have 
>> to do something about C++. As for everything else, we will share 
>> detailed information as we will have clearer plans about how XYZ fits 
>> into the general picture.
>>
>> As a meta-comment, your attitude is not promoting more openness; is 
>> actually having the opposite effect: if I cannot share an idea (e.g. 
>> having a mechanism to gate access to unsafe APIs) without having to 
>> pay a multi-email cost to keep you happy (and this stuff detracts 
>> time for everything else we have to do!), then I guess it's 
>> understandable if, next time, I will think twice before hitting "Send" ?
>>
>> I think the information we shared so far are more than enough to 
>> understand the broader picture of what we want to do; let me 
>> summarize it here to make it 100% clear, but please let's close this 
>> discussion down for now, and let's revisit once more info is actually 
>> available:
>>
>> * not all modules or jars are created equals - some require 'native' 
>> access, some don't
>> * module or jars which require native access can insert subtle 
>> crashes in your application
>> * there is no mechanism (at the moment) to distinguish between a 
>> regular module/jar and one which require native access
>> * as a result, an application which depends on 256 jars/modules 
>> doesn't know which one might blow it up
>> * _both_ JNI and the SystemABI API are functionalities that are 
>> _more_ privileged than, say "String::substring"
>> * I think it is time we look at some mechanism to separate between 
>> these two (very different) kind of modules/jar (mechanism TBD)
>>
>> Should this have been done on JNI from day one? Yes, of course. But I 
>> doubt that when JNI was created the level of understanding about what 
>> can go wrong when you start writing application that depends on 
>> hundreds of jars was as deep as it is now. Should such an effort have 
>> been part of Jigsaw? It could have been, of course, but there's only 
>> so much you can stuff under the same umbrella (if you want to produce 
>> actual deliverables, that is). As aspects of jigsaw were still being 
>> actively discussed past Java SE 9 [1, 2] - and are still actively 
>> being discussed _now_ [3] - offering clients some degree of 
>> protection against native libraries is yet another turn of the crank 
>> which fits in the general picture of making the Java world safer for 
>> everybody. The fact that it wasn't done before is a very weak 
>> argument in support of the status quo.
>>
>> [1] - https://openjdk.java.net/jeps/320
>> [2] - https://openjdk.java.net/jeps/335
>> [3] - https://bugs.openjdk.java.net/browse/JDK-8220607
>>
>> Maurizio
>>
>>
>>