VM crash with stack overflow: 25K load-barriers after split-if?

[Aleksey Shipilev]

Hi guys,

Our people have been running ZGC EA with our JDK 9-enabled Compiler.compiler test (cannot share it,
unfortunately), and it SEGVs with something that looks like a stack overflow error. It is
reproducible with current zgc/zgc in fastdebug. No VM diagnostics is available, but if you peek into
the core dump, then:

------------ 8< -----------------------------------------------------------------------------------

(gdb) info stack
#0  0x00007f9e456f6638 in TypeTuple::eq (this=0x7f9b22b6c518, t=0x7f9d7dd37dc8) at
/home/shade/trunks/zgc-zgc/src/hotspot/share/opto/type.cpp:1999
#1  0x00007f9e456f6dcd in Type::cmp (t1=0x7f9b22b6c518, t2=0x7f9d7dd37dc8) at
/home/shade/trunks/zgc-zgc/src/hotspot/share/opto/type.cpp:403
#2  0x00007f9e44ace71b in Dict::Insert (this=this at entry=0x7f9d6c001520,
key=key at entry=0x7f9b22b6c518, val=val at entry=0x7f9b22b6c518, replace=replace at entry=false)
    at /home/shade/trunks/zgc-zgc/src/hotspot/share/libadt/dict.cpp:208
#3  0x00007f9e456f8a31 in Type::hashcons (this=0x7f9b22b6c518) at
/home/shade/trunks/zgc-zgc/src/hotspot/share/opto/type.cpp:710
#4  0x00007f9e453366b3 in ProjNode::bottom_type (this=0x7f9d6cd44c48) at
/home/shade/trunks/zgc-zgc/src/hotspot/share/opto/multnode.cpp:107
#5  0x00007f9e452879a3 in LoadBarrierNode::bottom_type (this=0x7f9d82451da0) at
/home/shade/trunks/zgc-zgc/src/hotspot/share/opto/memnode.cpp:881
#6  0x00007f9e453366b3 in ProjNode::bottom_type (this=0x7f9d82451ea8) at
/home/shade/trunks/zgc-zgc/src/hotspot/share/opto/multnode.cpp:107
#7  0x00007f9e452879a3 in LoadBarrierNode::bottom_type (this=0x7f9d824522c8) at
/home/shade/trunks/zgc-zgc/src/hotspot/share/opto/memnode.cpp:881
#8  0x00007f9e453366b3 in ProjNode::bottom_type (this=0x7f9d824523d0) at
/home/shade/trunks/zgc-zgc/src/hotspot/share/opto/multnode.cpp:107
#9  0x00007f9e452879a3 in LoadBarrierNode::bottom_type (this=0x7f9d82452788) at
/home/shade/trunks/zgc-zgc/src/hotspot/share/opto/memnode.cpp:881
#10 0x00007f9e453366b3 in ProjNode::bottom_type (this=0x7f9d82452890) at
/home/shade/trunks/zgc-zgc/src/hotspot/share/opto/multnode.cpp:107
#11 0x00007f9e452879a3 in LoadBarrierNode::bottom_type (this=0x7f9d82452c48) at
/home/shade/trunks/zgc-zgc/src/hotspot/share/opto/memnode.cpp:881
...
... (this repeats around 25K times; notice: with different "this")
...
#25299 0x00007f9e45493996 in PhaseTransform::set_type_bottom (n=n at entry=0x7f9b21dc60b0,
this=0x7f9dc78a56e0) at /home/shade/trunks/zgc-zgc/src/hotspot/share/opto/phaseX.hpp:237
#25300 PhaseIterGVN::register_new_node_with_optimizer (this=0x7f9dc78a56e0,
n=n at entry=0x7f9b21dc60b0, orig=orig at entry=0x0) at
/home/shade/trunks/zgc-zgc/src/hotspot/share/opto/phaseX.cpp:1183

#25301 0x00007f9e455da70d in PhaseIdealLoop::register_new_node (this=this at entry=0x7f9dc78a6290,
n=n at entry=0x7f9b21dc60b0, blk=blk at entry=0x7f9b21dc5fa8)
    at /home/shade/trunks/zgc-zgc/src/hotspot/share/opto/split_if.cpp:268
#25302 0x00007f9e451974ea in PhaseIdealLoop::clone_load_barrier (this=this at entry=0x7f9dc78a6290,
lb=lb at entry=0x7f9b21dc5c98, ctl=ctl at entry=0x7f9b21dc5b68, mem=mem at entry=0x7f9d6cd453d0,
oop_in=<optimized out>)
    at /home/shade/trunks/zgc-zgc/src/hotspot/share/opto/loopopts.cpp:1208
#25303 0x00007f9e45197baa in PhaseIdealLoop::split_barrier_thru_phi (this=this at entry=0x7f9dc78a6290,
lb=lb at entry=0x7f9b21dc5c98) at /home/shade/trunks/zgc-zgc/src/hotspot/share/opto/loopopts.cpp:1281
#25304 0x00007f9e451996d6 in PhaseIdealLoop::optimize_load_barrier (this=0x7f9dc78a6290,
lb=0x7f9b21dc5c98, last_round=<optimized out>) at
/home/shade/trunks/zgc-zgc/src/hotspot/share/opto/loopopts.cpp:1450
#25305 0x00007f9e45199b88 in PhaseIdealLoop::split_if_with_blocks_post
(this=this at entry=0x7f9dc78a6290, n=n at entry=0x7f9b21dc5c98, last_round=last_round at entry=false)
    at /home/shade/trunks/zgc-zgc/src/hotspot/share/opto/loopopts.cpp:1724
#25306 0x00007f9e4519a5de in PhaseIdealLoop::split_if_with_blocks (this=this at entry=0x7f9dc78a6290,
visited=..., nstack=..., last_round=last_round at entry=false)
    at /home/shade/trunks/zgc-zgc/src/hotspot/share/opto/loopopts.cpp:1757
#25307 0x00007f9e4518fd69 in PhaseIdealLoop::build_and_optimize (this=0x7f9dc78a6290,
do_split_ifs=true, skip_loop_opts=<optimized out>, last_round=<optimized out>)
    at /home/shade/trunks/zgc-zgc/src/hotspot/share/opto/loopnode.cpp:2885
#25308 0x00007f9e44957924 in Compile::Optimize (this=this at entry=0x7f9dc78a7c20) at
/home/shade/trunks/zgc-zgc/src/hotspot/share/opto/compile.cpp:2290

#25309 0x00007f9e4495998a in Compile::Compile (this=0x7f9dc78a7c20, ci_env=<optimized out>,
compiler=0x7f9e3c1ca580, target=<optimized out>, osr_bci=<optimized out>, subsume_loads=<optimized
out>,
    do_escape_analysis=true, eliminate_boxing=true, directive=0x7f9e3c11cb70) at
/home/shade/trunks/zgc-zgc/src/hotspot/share/opto/compile.cpp:873

------------ 8< -----------------------------------------------------------------------------------

It maps to current code like this:

const Type *ProjNode::bottom_type() const {
  if (in(0) == NULL) return Type::TOP;
  return proj_type(in(0)->bottom_type());  <--- here
}

const Type *LoadBarrierNode::bottom_type() const {
  const Type **floadbarrier = (const Type
**)(Compile::current()->type_arena()->Amalloc_4((Number_of_Outputs)*sizeof(Type*)));
  const Type* val_t = in(Oop)->bottom_type(); <--- here
  floadbarrier[Control] = Type::CONTROL;
  floadbarrier[Memory] = Type::MEMORY;
  floadbarrier[Oop] = val_t;
  const Type* res = TypeTuple::make(Number_of_Outputs, floadbarrier);
  return res;
}

...but I am pretty sure the graph itself is foobar-ed and 25K load barriers somehow got chained
together by split-if? I can provide more diagnostics, but we are struggling to come up with a
reproducer.

Thanks,
-Aleksey