RFR: 8297138: UB leading to crash in Amalloc with optimized builds

[Thomas Stuefe]

On Wed, 23 Nov 2022 12:52:04 GMT, Afshin Zafari <duke at openjdk.org> wrote:

> 8297138: UB leading to crash in Amalloc with optimized builds

@afshin-zafari Good catch. This is not only UB, but most certainly breaks if any kind of malloc headers are used (e.g. if NMT is enabled). 

Could you please, just for Reviewer convenience, add a description of problem+patch in the PR text? I usually just copy the JBS issue text and, if needed, add comments describing the patch.

> I would have expected the fix here to be making the `free` debug_only as well. It is far from clear to me we want this code in a product build.

I agree with @dholmes-ora . I would prefer this to be explicitly debug-only with ifdefs. Even though in product builds UseMallocOnly is const false.

As a point of discussion, not for this RFE, I wonder whether we could get rid of `UseMallocOnly` completely. It is a constant hindrance when reworking Arenas. I'm curious about what we use it for. The only reason I can think of is to use buffer overrun detection inherent in os::malloc. But that could be added to Arena allocations as well. I attempted such a rewrite last year but stopped when the patch became too complex, and I did not know if anyone even cared for UseMallocOnly. Do people care?

-------------

PR: https://git.openjdk.org/jdk/pull/11320