JVM crash with stack overflow on 11.0.4 using ZGC

[Peter Booth]

I suggest running with
 -XX:+PrintCompilation, and also -XX:+TraceDeoptimization flags. If the crash is always Shem compiling the same method/Class then you could try flagging that to not be compiled, as a short term workaround 

Sent from my iPhone

> On Sep 27, 2019, at 11:52 AM, Barry Galster <Barry.Galster at imc.com> wrote:
> 
> Hello Per and ZGC experts,
> We have encountered a periodic crash which shares many attributes with this previous posting: http://mail.openjdk.java.net/pipermail/zgc-dev/2018-May/000336.html
> 
> 1. The stack is copied below along with the version information and relevant jvm flags.
> 2. The crash occurs during ~50% of the executions for the past week (we have not yet created a repeatable test harness).
> 3. We will attempt to reproduce the crash with jdk13 and appreciate any other recommendations.
> 
> Regards,
> Barry
> 
> --
> ZGC/memory JVM flags:
> -server -Xms64G -Xmx64G -Xmn3G -XX:+UnlockExperimentalVMOptions -XX:+UseZGC -XX:+AlwaysPreTouch -XX:ConcGCThreads=12 -XX:+UseLargePages -XX:LargePageSizeInBytes=2m -XX:ZPath=/dev/hugepages -XX:+UseCodeCacheFlushing -Xlog:gc*=info,gc+age=trace,safepoint=debug:file=gc.log:time,level,tags
> 
> openjdk version "11.0.4" 2019-07-16
> OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.4+11)
> OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.4+11, mixed mode)
> 
> Program terminated with signal 11, Segmentation fault.
> 
> #0  0x00007fc1557f9881 in _dl_update_slotinfo () from /lib64/ld-linux-x86-64.so.2
> #1  0x00007fc1557e8078 in update_get_addr () from /lib64/ld-linux-x86-64.so.2
> #2  0x00007fc1557fe928 in __tls_get_addr () from /lib64/ld-linux-x86-64.so.2
> #3  0x00007fc153e08e40 in LoadBarrierNode::bottom_type() const () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #4  0x00007fc153b57abf in ProjNode::bottom_type() const () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #5  0x00007fc153e08eba in LoadBarrierNode::bottom_type() const () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #6  0x00007fc153b57abf in ProjNode::bottom_type() const () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> ...
> ...
> these two lines repeat ~25k times until the bottom of the stack:
> 
> #25290 0x00007fc153b57abf in ProjNode::bottom_type() const () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25291 0x00007fc153e08eba in LoadBarrierNode::bottom_type() const () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25292 0x00007fc153b57abf in ProjNode::bottom_type() const () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25293 0x00007fc153e08eba in LoadBarrierNode::bottom_type() const () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25294 0x00007fc153b57abf in ProjNode::bottom_type() const () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25295 0x00007fc153c03503 in PhaseIterGVN::register_new_node_with_optimizer(Node*, Node*) () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25296 0x00007fc153e082c6 in clone_load_barrier(PhaseIdealLoop*, LoadBarrierNode*, Node*, Node*, Node*) () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25297 0x00007fc153e1225d in ZBarrierSetC2::loop_optimize_gc_barrier(PhaseIdealLoop*, Node*, bool) () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25298 0x00007fc153a5964f in PhaseIdealLoop::split_if_with_blocks(VectorSet&, Node_Stack&, bool) () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25299 0x00007fc153a53216 in PhaseIdealLoop::build_and_optimize(bool, bool, bool) () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25300 0x00007fc1535db4a3 in Compile::Optimize() () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25301 0x00007fc1535dc96a in Compile::Compile(ciEnv*, C2Compiler*, ciMethod*, int, bool, bool, bool, DirectiveSet*) () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25302 0x00007fc1534f26bc in C2Compiler::compile_method(ciEnv*, ciMethod*, int, DirectiveSet*) () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25303 0x00007fc1535e6a2d in CompileBroker::invoke_compiler_on_method(CompileTask*) () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25304 0x00007fc1535e81d8 in CompileBroker::compiler_thread_loop() () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25305 0x00007fc153d69423 in JavaThread::thread_main_inner() () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25306 0x00007fc153d696f5 in JavaThread::run() () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25307 0x00007fc153d653aa in Thread::call_run() () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25308 0x00007fc153badc9e in thread_native_entry(Thread*) () from /opt/openjdk-11.0.4/lib/server/libjvm.so
> #25309 0x00007fc155120dd5 in start_thread () from /lib64/libpthread.so.0
> #25310 0x00007fc154a34ead in clone () from /lib64/libc.so.6
> 
> Barry Galster
> Performance Engineer - Team Lead
> T +13122047574
> E Barry.Galster at imc.com<mailto:Barry.Galster at imc.com>
> 233 South Wacker Drive # 4250,
> Chicago, Illinois 60606, US
> [IMC Logo]<https://www.imc.com/us/>
> 
> [F]<https://www.facebook.com/IMCTrading>
> 
> 
> [t]<http://twitter.com/IMCTrading>
> 
> 
> [I]<https://www.instagram.com/imctrading/>
> 
> 
> [in]<https://www.linkedin.com/company/imc-financial-markets>
> 
> 
> imc.com<https://www.imc.com/us/>
> 
> 
> 
> ________________________________
> 
> The information in this e-mail is intended only for the person or entity to which it is addressed.
> 
> It may contain confidential and /or privileged material, the disclosure of which is prohibited. Any unauthorized copying, disclosure or distribution of the information in this email outside your company is strictly forbidden.
> 
> If you are not the intended recipient (or have received this email in error), please contact the sender immediately and permanently delete all copies of this email and any attachments from your computer system and destroy any hard copies. Although the information in this email has been compiled with great care, neither IMC nor any of its related entities shall accept any responsibility for any errors, omissions or other inaccuracies in this information or for the consequences thereof, nor shall it be bound in any way by the contents of this e-mail or its attachments.
> 
> Messages and attachments are scanned for all known viruses. Always scan attachments before opening them.