aarch64 call arranger misclassifies structures

[Владимир Козелков]

Hello!
Under the pull request <https://github.com/openjdk/jdk/pull/21041> I was
advised to write about new issues to this email.

I'm building my own port of the Panama project to Android, which uses
LibLLVM for native code generation, so I've been doing a lot of research
into how the calling convention works there.

Based on the data I received, I created my own classifier
<https://github.com/vova7878/PanamaPort/blob/master/AndroidPanama/src/main/java/com/v7878/foreign/_LLVMCallingConvention.java>
of structures for the arranger with an eye on LLVM (In most cases, it
calculates registers itself)

Initially my implementation rejected structures and sequences of size 0,
but then I added support for them back. Everything worked pretty well until
I dug into the godbolt tests and compared my implementation to the
reference implementation from OpenJDK.

It seems to me that
jdk.internal.foreign.abi.aarch64.TypeClass.isHomogeneousFloatAggregate
<https://github.com/openjdk/jdk/blob/260d4658aefe370d8994574c20057de07fd6f197/src/java.base/share/classes/jdk/internal/foreign/abi/aarch64/TypeClass.java#L85>
does not work correctly for union types and sequences with nested
GroupLayout. Additionally, I made examples in godbolt that show how it
determines HFA

C++ https://godbolt.org/z/fcsMvaKhd
C https://godbolt.org/z/er8Yev7ME

I don't have a PC on aarch64 to run tests on java, but it looks to me like
OpenJDK implementation is not correct for some cases
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20241006/22385593/attachment.htm>