RFR: 8340205: Native linker allows MemoryLayout consisting of only PaddingLayout [v5]

[Maurizio Cimadamore]

On Mon, 7 Oct 2024 15:48:46 GMT, Vladimir Kozelkov <duke at openjdk.org> wrote:

> > Does this restacking make things clearer?
> 
> I really like this new definition.
> 
> To what extent do these rules determine the compatibility of FFM calling conventions with the C calling convention? If I understand correctly, all layouts accepted by the linker must comply with the C calling convention, but linker must not allow all the cases that C allows?

I think the new rules define a bunch of rules that all layouts that model C type declarations should follow - but do so in broad strokes, still allowing the linker implementation to have the final say on what is and is not accepted.

The main problem with the current approach is that we are trying to define all C layouts in a platform agnostic fashion. But, as we discovered more and more over the last week or so, a lot of things which seemed "sensible" back when we wrote the javadoc, in reality rely on this or that platform/compiler assumption. And, given that the support for packed structs was already linker-dependent, it seems to make more sense to just say which things are "clearly wrong" (and then say that linker can reject more if it so wishes), rather than say which ones are "clearly correct" (and then say that linker can support more if it wants). It's a subtle change, but one that doesn't send us spinning trying to mimic complex C rules in where certain layouts might be allowed. For instance, the C standard disallows empty arrays. But empty arrays can be used as the last field of a struct (flexible array member) but only if they are preceded by some other field (and only if the struct is not nested in som
 e other struct)... as you can see, things get hairy pretty quickly, so I think it's probably better to say which general shape linker layouts should adhere to, while still leaving the linker a lot of say in what is/not correct.

> 
> I think it might be worth renaming the pull request and perhaps the issue to reflect that what's going on has moved well beyond the original bug report.

Agreed

-------------

PR Comment: https://git.openjdk.org/jdk/pull/21041#issuecomment-2397884234