[lworld] RFR: 8341757: Field layout computation allowing atomic and nullable flattening [v2]

[Dan Heidinga]

On Tue, 22 Oct 2024 14:45:28 GMT, Frederic Parain <fparain at openjdk.org> wrote:

>> src/hotspot/share/oops/instanceKlass.cpp line 1374:
>> 
>>> 1372:                 CLEAR_PENDING_EXCEPTION;
>>> 1373:             }
>>> 1374:             THROW_OOP(e());
>> 
>> If I understand this code correctly, an OutOfMemoryError here will make the class unusable, which is a bit surprising.  Is it possible to instead disable all layouts except the REFERENCE if an OOM is thrown here?
>
> Only the NULLABLE_ATOMIC_FLAT layout would need to be disabled. But if an OOM is thrown at this stage (class initialization), does it really worth the effort to try to recover the class? It is unlikely the application will be able to allocate an instance of this class anyway.

I'm OK with leaving the code as is - an OOM thrown during class initialization would poison the class anyways.  This is just one more case of that.

-------------

PR Review Comment: https://git.openjdk.org/valhalla/pull/1275#discussion_r1822904166