Concerns about signedness and ABI

[Maurizio Cimadamore]

Hi Andrew

On 19/07/2024 10:14, Andrew Haley wrote:
> On 7/17/24 19:10, Maurizio Cimadamore wrote:
>> I managed to come up with a reproducer, and I've filed this:
>>
>> https://bugs.openjdk.org/browse/JDK-8336664
>>
>> TBH, it is still not super clear to me how much this is a quirk in clang
>> and how much this is really what should be considered a "de facto 
>> standard".
>
> It's a known bug in clang:
>
> https://groups.google.com/g/x86-64-abi/c/h7FFh30oS3s/m/NDV4lCRQAQAJ
>
> This is x86-only, AFAIK. Other ABIs didn't fall into this trap. For
> example, AArch64 states quite explicitly that only the lower n bits in
> a register contain the argument.

I'm seeing this with clang x64 too, see reproducer here:

https://bugs.openjdk.org/browse/JDK-8336664


>
>> We are aware e.g. of 32 bit promotion of narrow arguments in variadic,
>> or prototype-less functions (as noted in the issue above), but other
>> than that, the ABI (and the C spec) is silent on this topic.
>>
>> It would be useful to know if sign extension is required in ABIs _other_
>> than SysV. Using compiler explorer, it seems like both gcc/clang on ARM
>> both apply the expected masking at the callee (unlike for x64/clang).
>> Same for Windows.
>
> I don't think there's any way to fix this without requiring the user of
> the downcall ABI to specify signedness.

I agree.

While expanding our horizon beyond SysV, we have found out (as 
documented in the issue linked above) that:

* RISCV64 requires narrow arguments to be zero/sign-extended by the 
caller. This behavior can run into similar issues as the one discussed here.
* ARM64 treats "char" as unsigned. But, since the ABI is clear in 
treating any unused upper bytes as garbage, an ARM64 compiler never 
relies on the upper bytes to be zeros (I've added some links to the bug 
above).

Combined with the fact that some of the restrictions we have around 
variadic argument passing are rather convoluted (e.g. we require the 
client of Linker to do the kind of argument promotion required by C), I 
think it would be an improvement to add sign information in value layouts.

Maurizio