[foreign] structs passing: windows vs. SystemV

[Jorn Vernee]

> 3) We could do *nothing* - that is, *never* pass a struct bigger than
> 64 bits on the stack (e.g. give an exception when that happens on
> Windows); if the user wants to pass such structs, it will have to
> allocate a Pointer and pass that instead

This is interesting, but doing that might make some people angry. In C 
the compiler always takes care of allocating the copy when passing 
larger than 64-bit structs, so it's reasonable to expect the same when 
using panama. (FWIW, it's not just smaller than 64-bit structs, it's 
exactly 8,16,32 or 64 bits; i.e. 24-bit structs are also passed in 
memory)

On the other hand it might offer some user-space opportunity for 
optimization where they elide the copy since they don't need the old 
value any more, but then again, that might also be a source of bugs; 
where users naïvely pass a pointer to their struct to a native function 
(without allocating a copy) and expect to get by-value semantics, and 
end up shooting themselves in the foot.

> 2) We could treat this as an ABI quirk - and add some custom code to
> the UniversalInvoker under WIndows, so that one strategy or the other
> is used depending on the struct size

This is currently what I'm using, and I feel that it's the most straight 
forward way to implement it. We can make the unbox-/box-value methods of 
UniversalNativeInvoker something that is implemented separately by every 
ABI. The only thing that would have to be added is a Scope argument 
which is closed after the call, so that unboxValue can use it to 
allocate a copy (in the current prototype I'm leaking the copy).

This approach also gives extra flexibility for other ports if they need 
to do something special for boxing and unboxing.

> 1) We could introduce a new recipe class - e.g. distinguish indirect
> struct pointers from normal INTEGER_ARGUMENT classes -
> UniversalInvoker can now disambiguate the two cases.

This could work I think, but it requires special casing in the 
UniversalNativeInvoker code which SysV has no need for. There's also 
some assertions being made in the current unboxing code which do not 
work for Windows [1]; the maximum number of bindings per argument is 
always 1. This conflicting assumption also makes me feel like 2. is a 
more solid approach, at least for the moment. Maybe if other ABIs use 
this form of argument passing as well we could look again at moving it 
to shared code.

Jorn

[1] : 
http://hg.openjdk.java.net/panama/dev/file/62c3c062d504/src/java.base/share/classes/jdk/internal/foreign/invokers/UniversalNativeInvoker.java#l173

Maurizio Cimadamore schreef op 2018-12-05 12:42:
> Hi,
> I'm coming back to this topic, as this is a source of issue when
> considering Windows port. More specifically, there is a difference in
> how structs are passed as parameters to native functions under the two
> ABIs:
> 
> *SystemV*: structs are recursively classified and decomposed into
> their fields - the fields are then passed separately in registers
> (integer/fp) or, if there's no more register space on stack.
> 
> *Windows*: structs up to 64 bits are passes in (integer) registers;
> bigger aggregates are passes in register, but _as pointers_ (to some
> storage that is caller-allocated); if there's no space on registers,
> again stack is used.
> 
> So, in essence, Windows ABI is unable to deal with structs bigger than
> 64bits - and has to resort to an indirection. This is a kind of a new
> beast that we have not encountered before in SystemV-land: on the one
> hand we are using a register, on the other hand we're using the
> register to store a pointer - an indirection to the thing we wanna
> pass.
> 
> There are several ways to deal with this:
> 
> 1) We could introduce a new recipe class - e.g. distinguish indirect
> struct pointers from normal INTEGER_ARGUMENT classes -
> UniversalInvoker can now disambiguate the two cases.
> 
> 2) We could treat this as an ABI quirk - and add some custom code to
> the UniversalInvoker under WIndows, so that one strategy or the other
> is used depending on the struct size
> 
> 3) We could do *nothing* - that is, *never* pass a struct bigger than
> 64 bits on the stack (e.g. give an exception when that happens on
> Windows); if the user wants to pass such structs, it will have to
> allocate a Pointer and pass that instead
> 
> I believe (2) is our plan of records; which makes sense, given the
> SystemABI API we are working on will make it easy to sweep these
> differences under the 'ABI rug'.
> 
> But (3) isn't as crazy as it sounds: after all, that's how we deal
> with arrays in native function signatures too - we don't support them
> and we ask the programmer to 'lower' them onto pointers (which is what
> the language does anyway). Also, if you have some automated tool
> generating all your API (like jextract), such tool could be made aware
> of the sharp edge, and always insert pointers when handling big
> structs in functions on Windows.
> 
> Of course that would also introduce differences in the extracted API
> between Windows and Linux/Max, so not all is rosy. But I thought it
> was worth at least considering this option, given that, with it, the
> existing ABI machinery can be reused almost in full.
> 
> Approach (1) seems to be the most consistent IMHO - we already have a
> CallingSequence::returnsInMemory() predicate, which is used to tell
> whether we need to allocate the return value onto some temporary
> storage (a pointer to which is returned in the first integer
> register). But we're missing its argument equivalent simply because
> SystemV doesn't happen to need it. For that reason, I think that
> generalizing over that concept so that every value passed in register
> can be an indirection could be an useful addition to UniversalInvoker.
> 
> Maurizio