RFR: 8156500: deadlock provoked by new stress test com/sun/jdi/OomDebugTest.java

[David Holmes]

On 30/06/2016 10:32 AM, Mandy Chung wrote:
>
>> On Jun 28, 2016, at 10:19 PM, David Holmes <david.holmes at oracle.com> wrote:
>>
>> So here is what I see has happened.
>>
>> Looking back at 9-b01, before we forced the initialization of InterruptedException and thus Throwable we find:
>>
>> 58 Initializing 'java/lang/Throwable' (0x0000000800002990)
>>
>> So Kim is right this was working by accident. It just seems that back then there was less memory required by the initialization of all the collection and other classes and so we didn't run into this problem.
>>
>> Post the InterruptedException change the initialization order made it unlikely an OOME could be encountered before Throwable was initialized (and after we have reached a point where we can throw without the VM crashing or instead doing a vm_exit_during_initialization).
>>
>> Post modules those collection classes, and others, are now done earlier again and before Throwable. And presumably their memory demands have increased.
>>
>
> This is the new primordial class added by modules that causes additional classes to be loaded early before initPhase1.
>
>   // The VM creates objects of this class.
>   initialize_class(vmSymbols::java_lang_reflect_Module(), CHECK);
>
>> Although we preallocate the OutOfMemoryError instances, and avoid executing any java code to do so, we can't necessarily** "throw" them until after Throwable is initialized. We now have a lot more initialization occurring before we init Throwable and so OOME is more likely and so it will fail as Kim observed.
>
>
> Would initializing java.lang.Throwable after java.lang.reflect.Module address this issue?  I don’t think I fully follow the problem Kim observed and below.

The earlier you initialize Throwable the earlier you can try to create 
an exception that has a backtrace. But there will always be a region of 
code where we can't throw an OOME with a backtrace because of the 
missing initialization of Throwable. So we can narrow that window by 
moving the initialization of Throwable (which in turn requires a whole 
bunch of collection classes - so the window has a fixed minimum size [ 
unless we do some creative restructuring of Throwable's static 
initialization]). My argument, which I think I've now convinced Kim of, 
is that we shouldn't be trying to throw an OOME with a stacktrace if 
Throwable has not been initialized - and that is where the change to 
gen_out_of_memory_error comes in. It is actually passed the 
pre-allocated OOME that has no backtrace and will never have a 
backtrace, and that is what should be thrown when Throwable has not been 
initialized.

Cheers,
David

>>
>> ** I say necessarily because I still believe it is the fact we attempt to fill in the stacktrace that leads to the dependency on Throwable being initialized, and we should be able to avoid that if we check the VM initialization state in gen_out_of_memory_error().
>
> Mandy
>