[foreign] RFR 8218153: Read/Write of Value layout should honor declared endianness

[Henry Jen]

On Feb 2, 2019, at 7:24 AM, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
> 
>> 
>>> * I'd like to understand more about DirectNativeInvoker limitations (mostly because this will carry over to linkToNative too) - what exactly goes wrong if we allow mismatching endianness there? Let me unpack this a bit: the direct invoker is only concerned in those calls that occur via registers only - why is endianness a factor here - since we are never storing into memory?
>>> 
>> DNI will call the JNI methods with the Java argument value. But when we specify a function signature with a different endian from platform, because the value is not going through Reference we won’t get the proper value. The EndianTest.java:322 illustrated such use case.
> I'm wondering what's the meaning of 'calling a function with different endianness' - I thought endianess had to do with mismatches between in-memory representation and in-register representation. When you extract a 'long' from memory, you need to know if the layout is LE or BE because, depending on your platform, you might need byte swap. But when you pass the same 'long' to a function, the long is already in a register (thanks to the VM) and you are probably copying it into another register, or on the stack - but as is. In other words, I guess I don't see how endianness would come to play in function calls. I know that, in our annotations, we allow endianness to be specified, but I'm not sure it makes sense talk about passing parameter to functions using a different endiannes?

Whenever multiple bytes are to be evaluated, endianness matters, both side need to agree on what it is. This is why the argument for layout to be explicit keeps coming up.

As long as we allow to specify endianness on function arguments, I think we need to honor that. From a language type perspective, it is expecting platform endian, which is why I argue that default should be platform endian if we allow such construction.

Cheers,
Henry