RFR: 8303002: Reject packed structs from linker [v5]

[Maurizio Cimadamore]

On Tue, 18 Apr 2023 13:45:57 GMT, Jorn Vernee <jvernee at openjdk.org> wrote:

>> This patch adds checks in AbstractLinker to reject packed structs and structs with excess padding (e.g. unnamed bitfields), since both of those are currently not supported, and the ABI/spec seems too vague to base support on.
>
> Jorn Vernee has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision:
> 
>  - fix TestIllegalLink
>  - Merge branch 'PR_21' into RejectPacked
>  - Merge branch 'PR_21' into RejectPacked
>  - add javadoc + fixes for trailing padding
>  - fix check for padding. Add check for trailing padding too
>  - Reject packed structs and structs with extra padding
>  - Check byte order of layouts passed to linker

src/java.base/share/classes/jdk/internal/foreign/abi/AbstractLinker.java line 120:

> 118:     private void checkLayoutsRecursive(MemoryLayout layout) {
> 119:         checkHasNaturalAlignment(layout);
> 120:         checkByteOrder(layout);

for uniformity, shoudn't this check be inside an `if (layout instanceof ValueLayout) ...` ?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/13164#discussion_r1170628266