RFC: 6898462: The escape analysis with G1 cause crash assertion src/share/vm/runtime/vframeArray.cpp:94

[John Rose]

On Nov 20, 2014, at 5:41 AM, Roland Westrelin <roland.westrelin at oracle.com> wrote:

> Thanks for the comments. I’ll work on your suggestions.
> 
> 
>> On Nov 19, 2014, at 12:19 AM, Vladimir Kozlov <vladimir.kozlov at oracle.com> wrote:
>> 
>> Good :) Really, I almost did not find any serious issues with these changes.
>> 
>> The only question I have: why not pop all interpreter frames corresponding to deoptimized compiled frame? You are trying to find the top popped frame in vframeArray.cpp. But how we guarantee that frames which are not popped are in correct state after deoptimization?
> 
> 
> That was John’s suggestion (in the chat room) or at least how I interpreted it:
> 
> "I like the frame-popping idea.  Given a bunch of frames to de-opt with OOME, you only need to pop until there are no more locals to materialize.”
> 
> But you’re right that objects A and B could have been scalar replaced. A has a reference to B. B is not reallocated. A is. Then A has a reference to null and some activation we don’t pop could try to access the non allocated B through A which would cause a NPE. So I agree that we should pop all frames. What do you think, John?

Popping all frames has a non-local effect:  You can kill an unbounded number of callers because of one callee failure.  This sort of near-death cascading failure inherently happens with VM errors (OOME and SOE).  My comment about popping "just enough" frames is a semi-futile attempt to limit the damage.

You could try to track the field assignments in Deoptimization::reassign_fields to detect which objects get assigned a null, and treat those objects as indirectly failed.  Then pop frames to eliminate all locals which directly or indirectly failed to reallocate.  This would require a transitive closure.  There are going to be bugs here.

So, I concur that popping all the frames is the best thing to do for now.

Please file a follow-up RFE to consider reserving memory for OOME exception processing.  (Cold, never-used memory would be reserved to recharge a thread's TLAB in some TBD gated manner, to allow catch clauses a better chance to complete.)  I'm not saying we have to do this, but we want to be ready if a use case comes up.

— John