[foreign] structs passing: windows vs. SystemV

[Maurizio Cimadamore]

Hi,
I'm coming back to this topic, as this is a source of issue when 
considering Windows port. More specifically, there is a difference in 
how structs are passed as parameters to native functions under the two ABIs:

*SystemV*: structs are recursively classified and decomposed into their 
fields - the fields are then passed separately in registers (integer/fp) 
or, if there's no more register space on stack.

*Windows*: structs up to 64 bits are passes in (integer) registers; 
bigger aggregates are passes in register, but _as pointers_ (to some 
storage that is caller-allocated); if there's no space on registers, 
again stack is used.

So, in essence, Windows ABI is unable to deal with structs bigger than 
64bits - and has to resort to an indirection. This is a kind of a new 
beast that we have not encountered before in SystemV-land: on the one 
hand we are using a register, on the other hand we're using the register 
to store a pointer - an indirection to the thing we wanna pass.

There are several ways to deal with this:

1) We could introduce a new recipe class - e.g. distinguish indirect 
struct pointers from normal INTEGER_ARGUMENT classes - UniversalInvoker 
can now disambiguate the two cases.

2) We could treat this as an ABI quirk - and add some custom code to the 
UniversalInvoker under WIndows, so that one strategy or the other is 
used depending on the struct size

3) We could do *nothing* - that is, *never* pass a struct bigger than 64 
bits on the stack (e.g. give an exception when that happens on Windows); 
if the user wants to pass such structs, it will have to allocate a 
Pointer and pass that instead

I believe (2) is our plan of records; which makes sense, given the 
SystemABI API we are working on will make it easy to sweep these 
differences under the 'ABI rug'.

But (3) isn't as crazy as it sounds: after all, that's how we deal with 
arrays in native function signatures too - we don't support them and we 
ask the programmer to 'lower' them onto pointers (which is what the 
language does anyway). Also, if you have some automated tool generating 
all your API (like jextract), such tool could be made aware of the sharp 
edge, and always insert pointers when handling big structs in functions 
on Windows.

Of course that would also introduce differences in the extracted API 
between Windows and Linux/Max, so not all is rosy. But I thought it was 
worth at least considering this option, given that, with it, the 
existing ABI machinery can be reused almost in full.

Approach (1) seems to be the most consistent IMHO - we already have a 
CallingSequence::returnsInMemory() predicate, which is used to tell 
whether we need to allocate the return value onto some temporary storage 
(a pointer to which is returned in the first integer register). But 
we're missing its argument equivalent simply because SystemV doesn't 
happen to need it. For that reason, I think that generalizing over that 
concept so that every value passed in register can be an indirection 
could be an useful addition to UniversalInvoker.

Maurizio