Reading binary data

[Maurizio Cimadamore]

On 06/11/2019 04:32, Samuel Audet wrote:
> Hi, Maurizio,
>
> We do need /something/ to replace sun.misc.Unsafe, so that's welcome, 
> but I'm concerned about it not actually being able to do the job. If I 
> understand correctly, it is not possible to create a MemorySegment 
> directly from an arbitrary long integer: We need to work around that 
> with Pointer. (Or did I miss a piece of the API somewhere?) However, 
> if I understand correctly, MemorySegment is scheduled to be released 
> before Pointer, which again, if I understand correctly, might not be 
> for a few more years. If that's the case, what are users meant to do 
> in meantime?

It is possible to create a memory address from a long:

http://hg.openjdk.java.net/panama/dev/file/0798bb838349/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/unsafe/ForeignUnsafe.java#l75

This is, currently (and for good reasons) an unsafe operation, so it is 
defined in a non-exported package which can be --add-exports(ed) if the 
user really need to do something like that. We also have ideas on how to 
allow operations such as these in a _safe_ way - that is, we could allow 
creation of 'random' addressed from long (including null), provided that 
we do not allow to safely dereference them. While this is not a 100% 
solution it would support NULL cleanly, as well as all cases where 
native API deal in opaque pointers.

Here there's a bigger, and very interesting theme, of how should an API 
containing both _safe_ and _unsafe_ parts should be defined in a way 
that (i) allow programmers to do what they have to do w/o being too 
punitive and (ii) prevents unsafe portions of the API to be repackaged 
and re-exported to innocuous clients operating in _safe_ mode. This is 
an hard trade-off to make, and it also doesn't help that, so far, JNI 
has always been treated as a _safe_ player.

I don't think the Pointer API has anything to create a Pointer from a 
long (unless you refer to the trick of writing a long in memory, and 
then reading it back as a Pointer).

Maurizio

>
> Samuel
>
> On 10/31/19 8:55 AM, Maurizio Cimadamore wrote:
>>
>> On 30/10/2019 20:29, Ty Young wrote:
>>> As an API user I would much rather have the APIs being integrated 
>>> with one another rather than completely separate APIs. Spending 
>>> hours trying to figure out how to read the data -only to use the 
>>> incorrect way that seems to return a valid value but in actuality 
>>> isn't- isn't going to be fun. 
>>
>> I think there is a misunderstanding here on what these API are meant 
>> to be.
>>
>> The memory access API is a low level API, it sits in the 
>> foreign-memaccess branch; that's what my talk (which you might have 
>> seen on youtube) speak about. It's meant to be a general off-heap 
>> API, with focus on safety and performances - kind of a replacement 
>> for Unsafe::get/putInt and friends.
>>
>> The Pointer API is what's available in the 'foreign' branch - and 
>> does what it says on the tin: it provides a Java abstraction which 
>> models a C source pointer type. The Pointer API is for language 
>> interop; the memory access API is lower level (in fact you can build 
>> the Pointer API on top of the memory access API).
>>
>> I'm sorry to hear your frustration in using the API (I assume the 
>> Pointer API?) but w/o understanding more of what you are trying to 
>> do, it's gonna be hard for us to offer any help or guidance.
>>
>> Cheers
>> Maurizio
>>