RFR: 8273608: Deadlock when jcmd of OnError attaches to itself [v4]

[David Holmes]

On 6/10/2021 11:39 am, Xin Liu wrote:
> On Tue, 5 Oct 2021 18:27:45 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:
> 
>>> Xin Liu has updated the pull request incrementally with one additional commit since the last revision:
>>>
>>>    Cleanup: remove a statment of debugging.
>>
>> test/hotspot/jtreg/runtime/ErrorHandling/TestOutOfMemoryErrorFromNIO.java line 25:
>>
>>> 23:
>>> 24: /*
>>> 25:  * @test TestOutOfMemoryErrorFromNIO
>>
>> I would rename the test. The point is not to test OOM from NIO, but to test XX:OnError with jcmd which attaches to the parent.
>>
>> You could do the same with any OOM, right? E.g. just start the VM with a very small MaxMetaspaceSize. You would not even need a main function for that, since the VM would not come up but throw a OOM right away,
>>
>> Proposal (but feel free to come up with something better): "TestOnErrorWithSelfAttachingJCmd"
> 
> I found HotSpot is very subtle when it comes to `MaxMetaspaceSize` and `-XX:AbortVMOnException=java.lang.OutOfMemoryError`
> 
> On my host, the watershed is 3M. If I give it 2M, it will ignore `AbortVMOnException`.
> 
> 
> java -XX:MaxMetaspaceSize=2M -XX:AbortVMOnException=java.lang.OutOfMemoryError
> Error occurred during initialization of VM
> OutOfMemoryError: Metaspace
> 
> 
> Flip code blocks here can solve this problem. is it intentional?

Yes it is intentional. If init has not completed you probably can't 
throw exceptions yet.

David
-----

> 
> diff --git a/src/hotspot/share/memory/metaspace.cpp b/src/hotspot/share/memory/metaspace.cpp
> index 74b5d1a0b30..efe430f7d3c 100644
> --- a/src/hotspot/share/memory/metaspace.cpp
> +++ b/src/hotspot/share/memory/metaspace.cpp
> @@ -974,15 +974,16 @@ void Metaspace::report_metadata_oome(ClassLoaderData* loader_data, size_t word_s
>           space_string);
>     }
> 
> -  if (!is_init_completed()) {
> -    vm_exit_during_initialization("OutOfMemoryError", space_string);
> -  }
> -
>     if (out_of_compressed_class_space) {
>       THROW_OOP(Universe::out_of_memory_error_class_metaspace());
>     } else {
>       THROW_OOP(Universe::out_of_memory_error_metaspace());
>     }
> +
> +  if (!is_init_completed()) {
> +    vm_exit_during_initialization("OutOfMemoryError", space_string);
> +  }
> +
>   }
> 
> -------------
> 
> PR: https://git.openjdk.java.net/jdk/pull/5590
>