8u102 NMethod sweeper related Hotspot crashes
Vitaly Davidovich
vitalyd at gmail.com
Mon Dec 12 20:19:37 UTC 2016
Great, thanks! Keep me posted :) And as mentioned, if you need any further
info, let me know (although I've included pretty much all I have at the
moment).
On Mon, Dec 12, 2016 at 3:15 PM, Vladimir Kozlov <vladimir.kozlov at oracle.com
> wrote:
> Converted to JDK bug:
>
> https://bugs.openjdk.java.net/browse/JDK-8171116
>
> Vladimir
>
> On 12/12/16 11:57 AM, Vitaly Davidovich wrote:
>
>> Thanks Vladimir.
>>
>> The segfault looks unrelated to this, so scratch that for now.
>>
>> I filed a JBS entry for the nmethod sweeper crashes though - internal id
>> is 9046080.
>>
>> I'm happy to provide any additional info that you guys may need, but I
>> put the full hs_err file contents for the 3 flavors of the crash we've
>> seen.
>>
>> Thanks
>>
>> On Mon, Dec 12, 2016 at 1:10 PM, Vladimir Kozlov
>> <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov at oracle.com>> wrote:
>>
>> If all in the same path (calls from
>> NMethodSweeper::process_nmethod()) then one JBS entry is enough.
>>
>> Vladimir
>>
>> On 12/12/16 10:05 AM, Vitaly Davidovich wrote:
>>
>> Hi Vladimir,
>>
>> There are now multiple crash reports around this code path, but
>> all with
>> a slightly different assertion error. Is one JBS entry with all
>> the
>> seen flavors sufficient or does each one need to be separate?
>> Unfortunately, turning off tiered compilation causes a segfault
>> although
>> I'm trying to get the full crash report to see if it's in the
>> same code
>> (path) or something else entirely.
>>
>> Thanks
>>
>> On Wed, Dec 7, 2016 at 1:20 PM, Vladimir Kozlov
>> <vladimir.kozlov at oracle.com <mailto:vladimir.kozlov at oracle.com>
>> <mailto:vladimir.kozlov at oracle.com
>> <mailto:vladimir.kozlov at oracle.com>>> wrote:
>>
>> All crashes have to have JBS entry. Please, file bug.
>>
>> JDK-8139595 is not related unfortunately. It fixed code
>> present only
>> in JDK 9.
>>
>> Thanks,
>> Vladimir
>>
>> On 12/7/16 10:13 AM, Vitaly Davidovich wrote:
>>
>> Vladimir,
>>
>> What do you think the next step should be? Is a JBS bug
>> entry
>> warranted?
>>
>> Thanks
>>
>> On Tue, Dec 6, 2016 at 1:12 PM Vitaly Davidovich
>> <vitalyd at gmail.com <mailto:vitalyd at gmail.com>
>> <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>>
>> <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>
>> <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>>>> wrote:
>>
>> Hi Vladimir,
>>
>> On Tue, Dec 6, 2016 at 12:57 PM, Vladimir Kozlov
>> <vladimir.kozlov at oracle.com
>> <mailto:vladimir.kozlov at oracle.com>
>> <mailto:vladimir.kozlov at oracle.com
>> <mailto:vladimir.kozlov at oracle.com>>
>> <mailto:vladimir.kozlov at oracle.com
>> <mailto:vladimir.kozlov at oracle.com>
>>
>> <mailto:vladimir.kozlov at oracle.com
>> <mailto:vladimir.kozlov at oracle.com>>>> wrote:
>>
>> Could be
>> https://bugs.openjdk.java.net/browse/JDK-8139595
>> <https://bugs.openjdk.java.net/browse/JDK-8139595>
>> <https://bugs.openjdk.java.net/browse/JDK-8139595
>> <https://bugs.openjdk.java.net/browse/JDK-8139595>> (not
>> backported to 8u).
>>
>> I can't find anything else with
>> remove_dependent_nmethod() on
>> call stack in JBS.
>>
>> It does seem like it could be related, but yeah,
>> it's unclear.
>> Hoping someone else here may have an idea.
>>
>> Thanks
>>
>>
>> Vladimir
>>
>> On 12/6/16 9:33 AM, Vitaly Davidovich wrote:
>>
>> Btw, for now I've advised the group hitting
>> these to
>> turn
>> off tiered
>> compilation to reduce code cache pressure
>> and thus
>> hopefully
>> avoid
>> whatever bug(s) is lurking here.
>>
>> I should've also mentioned that this doesn't
>> happen
>> all the
>> time, of
>> course, so there's no reliable repro.
>>
>> On Tue, Dec 6, 2016 at 12:02 PM Vitaly
>> Davidovich
>> <vitalyd at gmail.com
>> <mailto:vitalyd at gmail.com> <mailto:vitalyd at gmail.com
>> <mailto:vitalyd at gmail.com>>
>> <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>
>> <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>>>
>> <mailto:vitalyd at gmail.com
>> <mailto:vitalyd at gmail.com> <mailto:vitalyd at gmail.com
>> <mailto:vitalyd at gmail.com>>
>> <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>
>> <mailto:vitalyd at gmail.com <mailto:vitalyd at gmail.com>>>>> wrote:
>>
>> Hi guys,
>>
>> I have a couple of Hotspot crashes to
>> report - both
>> occur in nmethod
>> sweeping/flushing. I'm going to strip
>> down the
>> hs_err
>> content, but
>> let me know if there's something else
>> you need from
>> there. Are
>> these known? A quick google suggests
>> there are
>> some bugs
>> around
>> nmethod sweeping, but I couldn't find
>> anything
>> exactly
>> like this.
>>
>> Let me know if you need more info.
>>
>> Thanks
>>
>> *The first is:*
>>
>>
>> #
>> # A fatal error has been detected by the
>> Java
>> Runtime
>> Environment:
>> #
>> # Internal Error
>> (instanceKlass.cpp:1995),
>> pid=15444,
>> tid=0x00002b1fd0186700
>> # guarantee(val >= 0) failed: Underflow:
>> -1
>> #
>> # JRE version: Java(TM) SE Runtime
>> Environment
>> (8.0_102-b14) (build
>> 1.8.0_102-b14)
>> # Java VM: Java HotSpot(TM) 64-Bit Server
>> VM
>> (25.102-b14
>> mixed mode
>> linux-amd64 )
>> #
>> # If you would like to submit a bug
>> report,
>> please visit:
>> #
>> http://bugreport.java.com/bugreport/crash.jsp
>> <http://bugreport.java.com/bugreport/crash.jsp>
>> <http://bugreport.java.com/bugreport/crash.jsp
>> <http://bugreport.java.com/bugreport/crash.jsp>>
>> #
>>
>> --------------- T H R E A D
>> ---------------
>>
>> Current thread (0x00002b1f58056800):
>> JavaThread "C2
>> CompilerThread2" daemon [_thread_in_vm,
>> id=15533,
>>
>> stack(0x00002b1fd0086000,0x00002b1fd0187000)]
>>
>> Stack:
>> [0x00002b1fd0086000,0x00002b1fd0187000],
>> sp=0x00002b1fd0185580, free space=1021k
>> Native frames: (J=compiled Java code,
>> j=interpreted,
>> Vv=VM code,
>> C=native code)
>> V [libjvm.so+0xac52aa]
>> VMError::report_and_die()+0x2ba
>> V [libjvm.so+0x4fba22]
>> report_vm_error(char
>> const*,
>> int, char
>> const*, char const*)+0x62
>> V [libjvm.so+0x63e9b0]
>>
>> InstanceKlass::remove_dependent_nmethod(nmethod*,
>> bool)+0x110
>> V [libjvm.so+0x8e61b3]
>>
>> nmethod::flush_dependencies(BoolObjectClosure*)+0x93
>> V [libjvm.so+0x8ebd5b]
>>
>> nmethod::make_not_entrant_or_zombie(unsigned
>> int)+0x48b
>> V [libjvm.so+0xa2b0cd]
>> NMethodSweeper::process_nmetho
>> d(nmethod*)+0x27d
>> V [libjvm.so+0xa2b468]
>> NMethodSweeper::sweep_code_cache()+0x328
>> V [libjvm.so+0xa2b7d4]
>> NMethodSweeper::possibly_sweep()+0xb4
>> V [libjvm.so+0x4ae105]
>> CompileQueue::get()+0x15
>> V [libjvm.so+0x4b047b]
>> CompileBroker::compiler_thread_loop()+0x18b
>> V [libjvm.so+0xa73ce3]
>> JavaThread::thread_main_inner()+0x103
>> V [libjvm.so+0xa73e2c]
>> JavaThread::run()+0x11c
>> V [libjvm.so+0x9249c8]
>> java_start(Thread*)+0x108
>> C [libpthread.so.0+0x6b50]
>> start_thread+0xd0
>>
>> =>0x00002b1f58056800 JavaThread "C2
>> CompilerThread2" daemon
>> [_thread_in_vm, id=15533,
>> stack(0x00002b1fd0086000,0x00002b1fd0187000)]
>>
>> VM state:not at safepoint (normal
>> execution)
>> VM Mutex/Monitor currently owned by a
>> thread:
>> ([mutex/lock_event])
>> [0x0000000001ac5f40] CodeCache_lock -
>> owner thread:
>> 0x00002b1f58056800
>> [0x0000000001ac7240] CompiledIC_lock -
>> owner thread:
>> 0x00002b1f58056800
>>
>> Heap:
>> PSYoungGen total 16531968K, used
>> 13673471K
>> [0x00002b1b0b200000, 0x00002b1f49900000,
>> 0x00002b1f4b200000)
>> eden space 15297024K, 83% used
>>
>> [0x00002b1b0b200000,0x00002b1e
>> 19277f68,0x00002b1eb0c80000)
>> from space 1234944K, 69% used
>>
>> [0x00002b1eb0c80000,0x00002b1e
>> e5507eb0,0x00002b1efc280000)
>> to space 1198080K, 0% used
>>
>> [0x00002b1f00700000,0x00002b1f
>> 00700000,0x00002b1f49900000)
>> ParOldGen total 34891264K, used
>> 26246257K
>> [0x00002b128b200000, 0x00002b1adcb80000,
>> 0x00002b1b0b200000)
>> object space 34891264K, 75% used
>>
>> [0x00002b128b200000,0x00002b18
>> cd11c630,0x00002b1adcb80000)
>> Metaspace used 211158K, capacity
>> 610281K,
>> committed 610528K,
>> reserved 612352K
>>
>> Polling page: 0x00002b1281166000
>>
>> CodeCache: size=102400Kb used=71326Kb
>> max_used=81914Kb
>> free=31073Kb
>> bounds [0x00002b1284ab2000,
>> 0x00002b128aeb2000,
>> 0x00002b128aeb2000]
>> total_blobs=13619 nmethods=12436
>> adapters=1089
>> compilation: enabled
>>
>> *Here's the second:*
>>
>> #
>> # A fatal error has been detected by the
>> Java
>> Runtime
>> Environment:
>> #
>> # Internal Error
>> (instanceKlass.cpp:2018),
>> pid=83979,
>> tid=0x00002bb97feba700
>> # Error: ShouldNotReachHere()
>> #
>> # JRE version: Java(TM) SE Runtime
>> Environment
>> (8.0_102-b14) (build
>> 1.8.0_102-b14)
>> # Java VM: Java HotSpot(TM) 64-Bit Server
>> VM
>> (25.102-b14
>> mixed mode
>> linux-amd64 )
>> #
>> # If you would like to submit a bug
>> report,
>> please visit:
>> #
>> http://bugreport.java.com/bugreport/crash.jsp
>> <http://bugreport.java.com/bugreport/crash.jsp>
>> <http://bugreport.java.com/bugreport/crash.jsp
>> <http://bugreport.java.com/bugreport/crash.jsp>>
>> #
>>
>> --------------- T H R E A D
>> ---------------
>>
>> Current thread (0x00002bb8f40cf800):
>> JavaThread "C2
>> CompilerThread6" daemon [_thread_in_vm,
>> id=84042,
>>
>> stack(0x00002bb97fdba000,0x00002bb97febb000)]
>>
>> Stack:
>> [0x00002bb97fdba000,0x00002bb97febb000],
>> sp=0x00002bb97feb9710, free space=1021k
>> Native frames: (J=compiled Java code,
>> j=interpreted,
>> Vv=VM code,
>> C=native code)
>> V [libjvm.so+0xac52aa]
>> VMError::report_and_die()+0x2ba
>> V [libjvm.so+0x4fbd92]
>> report_should_not_reach_here(char const*,
>> int)+0x52
>> V [libjvm.so+0x63e8fb]
>>
>> InstanceKlass::remove_dependent_nmethod(nmethod*,
>> bool)+0x5b
>> V [libjvm.so+0x8e61b3]
>>
>> nmethod::flush_dependencies(BoolObjectClosure*)+0x93
>> V [libjvm.so+0x8ebd5b]
>>
>> nmethod::make_not_entrant_or_zombie(unsigned
>> int)+0x48b
>> V [libjvm.so+0xa2b0cd]
>> NMethodSweeper::process_nmetho
>> d(nmethod*)+0x27d
>> V [libjvm.so+0xa2b468]
>> NMethodSweeper::sweep_code_cache()+0x328
>> V [libjvm.so+0xa2b7d4]
>> NMethodSweeper::possibly_sweep()+0xb4
>> V [libjvm.so+0x4ae105]
>> CompileQueue::get()+0x15
>> V [libjvm.so+0x4b047b]
>> CompileBroker::compiler_thread_loop()+0x18b
>> V [libjvm.so+0xa73ce3]
>> JavaThread::thread_main_inner()+0x103
>> V [libjvm.so+0xa73e2c]
>> JavaThread::run()+0x11c
>> V [libjvm.so+0x9249c8]
>> java_start(Thread*)+0x108
>> C [libpthread.so.0+0x6b50]
>> start_thread+0xd0
>>
>> =>0x00002bb8f40cf800 JavaThread "C2
>> CompilerThread6" daemon
>> [_thread_in_vm, id=84042,
>> stack(0x00002bb97fdba000,0x00002bb97febb000)]
>> VM state:not at safepoint (normal
>> execution)
>>
>> VM Mutex/Monitor currently owned by a
>> thread:
>> ([mutex/lock_event])
>> [0x0000000001858f40] CodeCache_lock -
>> owner thread:
>> 0x00002bb8f40cf800
>> [0x000000000185a240] CompiledIC_lock -
>> owner thread:
>> 0x00002bb8f40cf800
>>
>> Heap:
>> PSYoungGen total 18567168K, used
>> 3149269K
>> [0x00002bb3cdf00000, 0x00002bb8e3400000,
>> 0x00002bb8e3400000)
>> eden space 15838208K, 5% used
>>
>> [0x00002bb3cdf00000,0x00002bb4
>> 02b26e00,0x00002bb794a00000)
>> from space 2728960K, 83% used
>>
>> [0x00002bb83ab00000,0x00002bb8
>> c624e6b0,0x00002bb8e1400000)
>> to space 2720768K, 0% used
>>
>> [0x00002bb794a00000,0x00002bb7
>> 94a00000,0x00002bb83ab00000)
>> ParOldGen total 23083520K, used
>> 21102913K
>> [0x00002ba9a3400000, 0x00002baf24280000,
>> 0x00002bb3cdf00000)
>> object space 23083520K, 91% used
>>
>> [0x00002ba9a3400000,0x00002bae
>> ab450478,0x00002baf24280000)
>> Metaspace used 167926K, capacity
>> 377784K,
>> committed 377856K, reserved 378880K
>>
>>
>>
>> Polling page: 0x00002ba999461000
>>
>> CodeCache: size=102400Kb used=72829Kb
>> max_used=81642Kb
>> free=29570Kb
>> bounds [0x00002ba99cdad000,
>> 0x00002ba9a31ad000,
>> 0x00002ba9a31ad000]
>> total_blobs=14843 nmethods=13668
>> adapters=1082
>> compilation: enabled
>>
>> --
>> Sent from my phone
>>
>> --
>> Sent from my phone
>>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/attachments/20161212/944f65a8/attachment-0001.html>
More information about the hotspot-compiler-dev
mailing list