8u102 NMethod sweeper related Hotspot crashes
Vladimir Kozlov
vladimir.kozlov at oracle.com
Mon Dec 12 20:15:36 UTC 2016
Converted to JDK bug:
https://bugs.openjdk.java.net/browse/JDK-8171116
Vladimir
On 12/12/16 11:57 AM, Vitaly Davidovich wrote:
> Thanks Vladimir.
>
> The segfault looks unrelated to this, so scratch that for now.
>
> I filed a JBS entry for the nmethod sweeper crashes though - internal id
> is 9046080.
>
> I'm happy to provide any additional info that you guys may need, but I
> put the full hs_err file contents for the 3 flavors of the crash we've seen.
>
> Thanks
>
> On Mon, Dec 12, 2016 at 1:10 PM, Vladimir Kozlov
> <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov at oracle.com>> wrote:
>
> If all in the same path (calls from
> NMethodSweeper::process_nmethod()) then one JBS entry is enough.
>
> Vladimir
>
> On 12/12/16 10:05 AM, Vitaly Davidovich wrote:
>
> Hi Vladimir,
>
> There are now multiple crash reports around this code path, but
> all with
> a slightly different assertion error. Is one JBS entry with all the
> seen flavors sufficient or does each one need to be separate?
> Unfortunately, turning off tiered compilation causes a segfault
> although
> I'm trying to get the full crash report to see if it's in the
> same code
> (path) or something else entirely.
>
> Thanks
>
> On Wed, Dec 7, 2016 at 1:20 PM, Vladimir Kozlov
> <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov at oracle.com>
> <mailto:vladimir.kozlov at oracle.com
> <mailto:vladimir.kozlov at oracle.com>>> wrote:
>
> All crashes have to have JBS entry. Please, file bug.
>
> JDK-8139595 is not related unfortunately. It fixed code
> present only
> in JDK 9.
>
> Thanks,
> Vladimir
>
> On 12/7/16 10:13 AM, Vitaly Davidovich wrote:
>
> Vladimir,
>
> What do you think the next step should be? Is a JBS bug
> entry
> warranted?
>
> Thanks
>
> On Tue, Dec 6, 2016 at 1:12 PM Vitaly Davidovich
> <vitalyd at gmail.com <mailto:vitalyd at gmail.com>
> <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>>
> <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>
> <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>>>> wrote:
>
> Hi Vladimir,
>
> On Tue, Dec 6, 2016 at 12:57 PM, Vladimir Kozlov
> <vladimir.kozlov at oracle.com
> <mailto:vladimir.kozlov at oracle.com>
> <mailto:vladimir.kozlov at oracle.com
> <mailto:vladimir.kozlov at oracle.com>>
> <mailto:vladimir.kozlov at oracle.com
> <mailto:vladimir.kozlov at oracle.com>
>
> <mailto:vladimir.kozlov at oracle.com
> <mailto:vladimir.kozlov at oracle.com>>>> wrote:
>
> Could be
> https://bugs.openjdk.java.net/browse/JDK-8139595
> <https://bugs.openjdk.java.net/browse/JDK-8139595>
> <https://bugs.openjdk.java.net/browse/JDK-8139595
> <https://bugs.openjdk.java.net/browse/JDK-8139595>> (not
> backported to 8u).
>
> I can't find anything else with
> remove_dependent_nmethod() on
> call stack in JBS.
>
> It does seem like it could be related, but yeah,
> it's unclear.
> Hoping someone else here may have an idea.
>
> Thanks
>
>
> Vladimir
>
> On 12/6/16 9:33 AM, Vitaly Davidovich wrote:
>
> Btw, for now I've advised the group hitting
> these to
> turn
> off tiered
> compilation to reduce code cache pressure
> and thus
> hopefully
> avoid
> whatever bug(s) is lurking here.
>
> I should've also mentioned that this doesn't
> happen
> all the
> time, of
> course, so there's no reliable repro.
>
> On Tue, Dec 6, 2016 at 12:02 PM Vitaly
> Davidovich
> <vitalyd at gmail.com
> <mailto:vitalyd at gmail.com> <mailto:vitalyd at gmail.com
> <mailto:vitalyd at gmail.com>>
> <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>
> <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>>>
> <mailto:vitalyd at gmail.com
> <mailto:vitalyd at gmail.com> <mailto:vitalyd at gmail.com
> <mailto:vitalyd at gmail.com>>
> <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>
> <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>>>>> wrote:
>
> Hi guys,
>
> I have a couple of Hotspot crashes to
> report - both
> occur in nmethod
> sweeping/flushing. I'm going to strip
> down the
> hs_err
> content, but
> let me know if there's something else
> you need from
> there. Are
> these known? A quick google suggests
> there are
> some bugs
> around
> nmethod sweeping, but I couldn't find
> anything
> exactly
> like this.
>
> Let me know if you need more info.
>
> Thanks
>
> *The first is:*
>
>
> #
> # A fatal error has been detected by the
> Java
> Runtime
> Environment:
> #
> # Internal Error (instanceKlass.cpp:1995),
> pid=15444,
> tid=0x00002b1fd0186700
> # guarantee(val >= 0) failed: Underflow: -1
> #
> # JRE version: Java(TM) SE Runtime
> Environment
> (8.0_102-b14) (build
> 1.8.0_102-b14)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM
> (25.102-b14
> mixed mode
> linux-amd64 )
> #
> # If you would like to submit a bug report,
> please visit:
> #
> http://bugreport.java.com/bugreport/crash.jsp
> <http://bugreport.java.com/bugreport/crash.jsp>
> <http://bugreport.java.com/bugreport/crash.jsp
> <http://bugreport.java.com/bugreport/crash.jsp>>
> #
>
> --------------- T H R E A D
> ---------------
>
> Current thread (0x00002b1f58056800):
> JavaThread "C2
> CompilerThread2" daemon [_thread_in_vm,
> id=15533,
>
> stack(0x00002b1fd0086000,0x00002b1fd0187000)]
>
> Stack:
> [0x00002b1fd0086000,0x00002b1fd0187000],
> sp=0x00002b1fd0185580, free space=1021k
> Native frames: (J=compiled Java code,
> j=interpreted,
> Vv=VM code,
> C=native code)
> V [libjvm.so+0xac52aa]
> VMError::report_and_die()+0x2ba
> V [libjvm.so+0x4fba22]
> report_vm_error(char
> const*,
> int, char
> const*, char const*)+0x62
> V [libjvm.so+0x63e9b0]
>
> InstanceKlass::remove_dependent_nmethod(nmethod*,
> bool)+0x110
> V [libjvm.so+0x8e61b3]
>
> nmethod::flush_dependencies(BoolObjectClosure*)+0x93
> V [libjvm.so+0x8ebd5b]
>
> nmethod::make_not_entrant_or_zombie(unsigned
> int)+0x48b
> V [libjvm.so+0xa2b0cd]
> NMethodSweeper::process_nmethod(nmethod*)+0x27d
> V [libjvm.so+0xa2b468]
> NMethodSweeper::sweep_code_cache()+0x328
> V [libjvm.so+0xa2b7d4]
> NMethodSweeper::possibly_sweep()+0xb4
> V [libjvm.so+0x4ae105]
> CompileQueue::get()+0x15
> V [libjvm.so+0x4b047b]
> CompileBroker::compiler_thread_loop()+0x18b
> V [libjvm.so+0xa73ce3]
> JavaThread::thread_main_inner()+0x103
> V [libjvm.so+0xa73e2c]
> JavaThread::run()+0x11c
> V [libjvm.so+0x9249c8]
> java_start(Thread*)+0x108
> C [libpthread.so.0+0x6b50]
> start_thread+0xd0
>
> =>0x00002b1f58056800 JavaThread "C2
> CompilerThread2" daemon
> [_thread_in_vm, id=15533,
> stack(0x00002b1fd0086000,0x00002b1fd0187000)]
>
> VM state:not at safepoint (normal execution)
> VM Mutex/Monitor currently owned by a
> thread:
> ([mutex/lock_event])
> [0x0000000001ac5f40] CodeCache_lock -
> owner thread:
> 0x00002b1f58056800
> [0x0000000001ac7240] CompiledIC_lock -
> owner thread:
> 0x00002b1f58056800
>
> Heap:
> PSYoungGen total 16531968K, used
> 13673471K
> [0x00002b1b0b200000, 0x00002b1f49900000,
> 0x00002b1f4b200000)
> eden space 15297024K, 83% used
>
> [0x00002b1b0b200000,0x00002b1e19277f68,0x00002b1eb0c80000)
> from space 1234944K, 69% used
>
> [0x00002b1eb0c80000,0x00002b1ee5507eb0,0x00002b1efc280000)
> to space 1198080K, 0% used
>
> [0x00002b1f00700000,0x00002b1f00700000,0x00002b1f49900000)
> ParOldGen total 34891264K, used
> 26246257K
> [0x00002b128b200000, 0x00002b1adcb80000,
> 0x00002b1b0b200000)
> object space 34891264K, 75% used
>
> [0x00002b128b200000,0x00002b18cd11c630,0x00002b1adcb80000)
> Metaspace used 211158K, capacity
> 610281K,
> committed 610528K,
> reserved 612352K
>
> Polling page: 0x00002b1281166000
>
> CodeCache: size=102400Kb used=71326Kb
> max_used=81914Kb
> free=31073Kb
> bounds [0x00002b1284ab2000,
> 0x00002b128aeb2000,
> 0x00002b128aeb2000]
> total_blobs=13619 nmethods=12436
> adapters=1089
> compilation: enabled
>
> *Here's the second:*
>
> #
> # A fatal error has been detected by the
> Java
> Runtime
> Environment:
> #
> # Internal Error (instanceKlass.cpp:2018),
> pid=83979,
> tid=0x00002bb97feba700
> # Error: ShouldNotReachHere()
> #
> # JRE version: Java(TM) SE Runtime
> Environment
> (8.0_102-b14) (build
> 1.8.0_102-b14)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM
> (25.102-b14
> mixed mode
> linux-amd64 )
> #
> # If you would like to submit a bug report,
> please visit:
> #
> http://bugreport.java.com/bugreport/crash.jsp
> <http://bugreport.java.com/bugreport/crash.jsp>
> <http://bugreport.java.com/bugreport/crash.jsp
> <http://bugreport.java.com/bugreport/crash.jsp>>
> #
>
> --------------- T H R E A D
> ---------------
>
> Current thread (0x00002bb8f40cf800):
> JavaThread "C2
> CompilerThread6" daemon [_thread_in_vm,
> id=84042,
>
> stack(0x00002bb97fdba000,0x00002bb97febb000)]
>
> Stack:
> [0x00002bb97fdba000,0x00002bb97febb000],
> sp=0x00002bb97feb9710, free space=1021k
> Native frames: (J=compiled Java code,
> j=interpreted,
> Vv=VM code,
> C=native code)
> V [libjvm.so+0xac52aa]
> VMError::report_and_die()+0x2ba
> V [libjvm.so+0x4fbd92]
> report_should_not_reach_here(char const*,
> int)+0x52
> V [libjvm.so+0x63e8fb]
>
> InstanceKlass::remove_dependent_nmethod(nmethod*,
> bool)+0x5b
> V [libjvm.so+0x8e61b3]
>
> nmethod::flush_dependencies(BoolObjectClosure*)+0x93
> V [libjvm.so+0x8ebd5b]
>
> nmethod::make_not_entrant_or_zombie(unsigned
> int)+0x48b
> V [libjvm.so+0xa2b0cd]
> NMethodSweeper::process_nmethod(nmethod*)+0x27d
> V [libjvm.so+0xa2b468]
> NMethodSweeper::sweep_code_cache()+0x328
> V [libjvm.so+0xa2b7d4]
> NMethodSweeper::possibly_sweep()+0xb4
> V [libjvm.so+0x4ae105]
> CompileQueue::get()+0x15
> V [libjvm.so+0x4b047b]
> CompileBroker::compiler_thread_loop()+0x18b
> V [libjvm.so+0xa73ce3]
> JavaThread::thread_main_inner()+0x103
> V [libjvm.so+0xa73e2c]
> JavaThread::run()+0x11c
> V [libjvm.so+0x9249c8]
> java_start(Thread*)+0x108
> C [libpthread.so.0+0x6b50]
> start_thread+0xd0
>
> =>0x00002bb8f40cf800 JavaThread "C2
> CompilerThread6" daemon
> [_thread_in_vm, id=84042,
> stack(0x00002bb97fdba000,0x00002bb97febb000)]
> VM state:not at safepoint (normal execution)
>
> VM Mutex/Monitor currently owned by a
> thread:
> ([mutex/lock_event])
> [0x0000000001858f40] CodeCache_lock -
> owner thread:
> 0x00002bb8f40cf800
> [0x000000000185a240] CompiledIC_lock -
> owner thread:
> 0x00002bb8f40cf800
>
> Heap:
> PSYoungGen total 18567168K, used
> 3149269K
> [0x00002bb3cdf00000, 0x00002bb8e3400000,
> 0x00002bb8e3400000)
> eden space 15838208K, 5% used
>
> [0x00002bb3cdf00000,0x00002bb402b26e00,0x00002bb794a00000)
> from space 2728960K, 83% used
>
> [0x00002bb83ab00000,0x00002bb8c624e6b0,0x00002bb8e1400000)
> to space 2720768K, 0% used
>
> [0x00002bb794a00000,0x00002bb794a00000,0x00002bb83ab00000)
> ParOldGen total 23083520K, used
> 21102913K
> [0x00002ba9a3400000, 0x00002baf24280000,
> 0x00002bb3cdf00000)
> object space 23083520K, 91% used
>
> [0x00002ba9a3400000,0x00002baeab450478,0x00002baf24280000)
> Metaspace used 167926K, capacity
> 377784K,
> committed 377856K, reserved 378880K
>
>
>
> Polling page: 0x00002ba999461000
>
> CodeCache: size=102400Kb used=72829Kb
> max_used=81642Kb
> free=29570Kb
> bounds [0x00002ba99cdad000,
> 0x00002ba9a31ad000,
> 0x00002ba9a31ad000]
> total_blobs=14843 nmethods=13668
> adapters=1082
> compilation: enabled
>
> --
> Sent from my phone
>
> --
> Sent from my phone
>
>
>
More information about the hotspot-compiler-dev
mailing list