[foreign] RFR 8218153: Read/Write of Value layout should honor declared endianness

[Henry Jen]

> On Feb 28, 2019, at 3:55 PM, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
> 
> Looked at the jextract patch too.
> 
> I'm a bit uneasy with the approach; I'll try to explain why.

I did say it is preliminary and I am looking for comment. The issue you caught also is why I am reserved with the test approach, as it depends on withEndianness is correctly implemented as expected.

> 
> You basically convert the struct layout and the var layout _after the fact_; that is, we first create layout that are endianness-less. Then, just before we write them out, we add endianness back in.
> 
> This has some bad consequences - as, I think, it will fail for all nested things. I think that, the fact that we are using symbolic references (Unresolved layouts) is saving you a little bit here.
> 
> e.g.
> 
> u64:${foo}
> 
> will work, because, when we will have to replace the actual layout of 'foo' into the above, we will find something that does have an endianness.
> 
> But that's mostly an accident. You can have pointer to arrays, too:
> 
> u64:[5 i32]
> 
> And in this case, withEndianness will do nothing for these.
> 

A-ha, I was expecting withEndianness will make all components within a layout with any element that is no-endianness to be replaced.

That means this example should work as well. So I considered this a bug and just did a jshell test to verify it.

However, I realize that is probably controversial as apparently you think this is correct behavior, and I think that’s is reasonable expectation as well, since that’s a pointee type, not really a part of address itself.

> I think the endianness should be computed on the fly, by LayoutUtils. And probably there has to be some state info which tells you whether you are generating a function layout (in which case you want to endianness).
> 
> But I think LayoutUtils should always push the endianness info down at the value components - that is, when it finds a C int type, it will figure out what is the platform endianness, and generate either >i32 or <i32 - assuming you are not in a toplevel function context.
> 

I’ll look into that.

Cheers,
Henry