RFR: 8269537: memset() is called after operator new [v2]

[Leo Korinth]

On Tue, 7 Sep 2021 23:29:10 GMT, Ioi Lam <iklam at openjdk.org> wrote:

>> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   First update
>>   
>>   * Change backing type of ResourceObj::allocation_type to be u8. Also remove no longer needed mask and explicit zero value of STACK_OR_EMBEDDED value.
>>   
>>   * Now setting allocation type with set_type() with assert.
>
> src/hotspot/share/memory/allocation.hpp line 439:
> 
>> 437:   void* operator new(size_t size, const std::nothrow_t& nothrow_constant) throw() {
>> 438:       address res = (address)resource_allocate_bytes(size, AllocFailStrategy::RETURN_NULL);
>> 439:       DEBUG_ONLY(if (res != NULL) _thread_last_allocated = RESOURCE_AREA;)
> 
> Maybe we should also guard against the possibility of nested allocations, which may trash `_thread_last_allocated`?
> 
> 
> #define PUSH_RESOURCE_OBJ_ALLOC_TYPE(t) \
>   assert(_thread_last_allocated == STACK_OR_EMBEDDED, "must not be nested"); \
>   DEBUG_ONLY(_thread_last_allocated = t); \
> 
> ...
>   if (res != NULL) {
>     PUSH_RESOURCE_OBJ_ALLOC_TYPE(RESOURCE_AREA);
>   }
> 
> 
> Similarly, the `ResourceObj` constructor should use a corresponding `POP_RESOURCE_OBJ_ALLOC_TYPE` macro.

I added a `set_type` method that ensures that the `_thread_last_allocated` always transition over a `STACK_OR_EMBEDDED`. I did *not* create a PUSH/POP macro pair because i believe it would give the false impression that we are doing a stack operation. 

Other than that I also made `allocation_type` use a `u8` as backing type. I also removed the now unused `allocation_mask` and the now unimportant detail that `STACK_OR_EMBEDDED = 0`.

-------------

PR: https://git.openjdk.java.net/jdk/pull/5387