[foreign-jextract] RFR: 8262851: jextract crashes with "Cannot compute size of a layout which is, or depends on a sequence layout with unspecified size"

[Maurizio Cimadamore]

On Thu, 4 Mar 2021 10:50:22 GMT, Athijegannathan Sundararajan <sundar at openjdk.org> wrote:

>> src/jdk.incubator.jextract/share/classes/jdk/internal/jextract/impl/StructBuilder.java line 140:
>> 
>>> 138:     public void addVar(String javaName, String nativeName, MemoryLayout layout, Class<?> type) {
>>> 139:         try {
>>> 140:             structLayout.byteOffset(elementPaths(nativeName));
>> 
>> Now that I think about it - calling byteSize on the structLayout should be enough. I also think that, if we do this, we can remove the logic in OutputFactory which checks whether size is available.
>
> hmm.. byteSize() does not result in exception being thrown (the previous struct field has undermined size because of unsized array at the end. but current offending field has proper size). Am I missing something?

I mean byteSize on `structLayout`. If there is any variable-length array, structLayout (which is supposedly storing the _enclosing_ layout of all nested anon structs inside it as well) should have its size undefined too.

-------------

PR: https://git.openjdk.java.net/panama-foreign/pull/464