RFR: 8156500: deadlock provoked by new stress test com/sun/jdi/OomDebugTest.java

[Mandy Chung]

> On Jun 28, 2016, at 10:19 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> So here is what I see has happened.
> 
> Looking back at 9-b01, before we forced the initialization of InterruptedException and thus Throwable we find:
> 
> 58 Initializing 'java/lang/Throwable' (0x0000000800002990)
> 
> So Kim is right this was working by accident. It just seems that back then there was less memory required by the initialization of all the collection and other classes and so we didn't run into this problem.
> 
> Post the InterruptedException change the initialization order made it unlikely an OOME could be encountered before Throwable was initialized (and after we have reached a point where we can throw without the VM crashing or instead doing a vm_exit_during_initialization).
> 
> Post modules those collection classes, and others, are now done earlier again and before Throwable. And presumably their memory demands have increased.
> 

This is the new primordial class added by modules that causes additional classes to be loaded early before initPhase1.

  // The VM creates objects of this class.
  initialize_class(vmSymbols::java_lang_reflect_Module(), CHECK);

> Although we preallocate the OutOfMemoryError instances, and avoid executing any java code to do so, we can't necessarily** "throw" them until after Throwable is initialized. We now have a lot more initialization occurring before we init Throwable and so OOME is more likely and so it will fail as Kim observed.


Would initializing java.lang.Throwable after java.lang.reflect.Module address this issue?  I don’t think I fully follow the problem Kim observed and below.

> 
> ** I say necessarily because I still believe it is the fact we attempt to fill in the stacktrace that leads to the dependency on Throwable being initialized, and we should be able to avoid that if we check the VM initialization state in gen_out_of_memory_error().

Mandy