RFR: 8303002: Reject packed structs from linker [v5]

[Maurizio Cimadamore]

On Tue, 18 Apr 2023 13:45:57 GMT, Jorn Vernee <jvernee at openjdk.org> wrote:

>> This patch adds checks in AbstractLinker to reject packed structs and structs with excess padding (e.g. unnamed bitfields), since both of those are currently not supported, and the ABI/spec seems too vague to base support on.
>
> Jorn Vernee has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision:
> 
>  - fix TestIllegalLink
>  - Merge branch 'PR_21' into RejectPacked
>  - Merge branch 'PR_21' into RejectPacked
>  - add javadoc + fixes for trailing padding
>  - fix check for padding. Add check for trailing padding too
>  - Reject packed structs and structs with extra padding
>  - Check byte order of layouts passed to linker

src/java.base/share/classes/java/lang/foreign/Linker.java line 202:

> 200:  * Due to limited ABI specification coverage, all the native linker implementations limit the function
> 201:  * descriptors that they support to those that contain only so-called <em>canonical</em> layouts. These layouts
> 202:  * have the following restrictions:

I think it's better to phrase this more or less as:

A canonical layout has the following characteristics:
* Its alignment constraint is equal to its natural alignment
* If the layout is a value layout (linkplain), its byte order matches that of the platform in which the JVM is executing (link to nativeOrder())
*  If the layout is a group layout (linkplain), it must not contain more padding layout (linkplain) elements than those strictly necessary to satisfy the alignment constraints of the non-padding elements of the group layout.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/13164#discussion_r1170624323