crash in JvmtiExport::post_compiled_method_load
JC Beyler
jcbeyler at google.com
Wed May 9 17:36:51 UTC 2018
Not that this helps but I was trying to see if I could get a repro for this
and understand a bit more how things work. I failed to do so but here was
what I did in case it helps get closer to an actual repro. I did a code
that had a lot of methods and reduced the various knobs to get to a case
where the JVM would keep loading/unloading methods:
final class TestCompileEvent {
public long cnt;
public static void main(String[] args) {
TestCompileEvent elem = new TestAOT();
for (int i = 0; i < 20000; i++) {
elem.foo0();
elem.foo1();
....
elem.foo3999();
}
System.out.println(elem.cnt);
}
where each foo method something like:
private void foo2() {
for (int i = 0; i < 2; i++) {
cnt += 2 + i / (2 + i + 1) + (2 * (i + 3.14));
}
}
The knobs I used were:
-XX:InitialCodeCacheSize=4k -XX:ReservedCodeCacheSize=2000k
-XX:CodeCacheMinimumUseSpace=1k -XX:CodeCacheExpansionSize=32k
-XX:-SegmentedCodeCache -XX:CompileThreshold=1 -XX:-Inline
This with a JVMTI agent logging compiled method load/unload seemed to keep
the code in a state of continuous loading/unloading of the various
methods. But I did not see the bug show up. I wonder if I need to add some
allocations in the methods and reduce the Xms/Xmx to get the GC to start
freaking out a bit.
Anyway, that was my analysis of this. Perhaps it helps?
Jc
On Wed, May 9, 2018 at 4:09 AM serguei.spitsyn at oracle.com <
serguei.spitsyn at oracle.com> wrote:
> Okay.
> More details on this.
>
> This is the nmethod locking api:
>
> / Locks an nmethod so its code will not get removed and it will not
> // be made into a zombie, even if it is a not_entrant method. After the
> // nmethod becomes a zombie, if CompiledMethodUnload event processing
> // needs to be done, then lock_nmethod() is used directly to keep the
> // generated code from being reused too early.
> class nmethodLocker : public StackObj {
> CompiledMethod* _nm;
>
> public:
>
> // note: nm can be NULL
> // Only JvmtiDeferredEvent::compiled_method_unload_event()
> // should pass zombie_ok == true.
> static void lock_nmethod(CompiledMethod* nm, bool zombie_ok = false);
> static void unlock_nmethod(CompiledMethod* nm); // (ditto)
>
> nmethodLocker(address pc); // derive nm from pc
> nmethodLocker(nmethod *nm) { _nm = nm; lock_nmethod(_nm); }
> nmethodLocker(CompiledMethod *nm) {
> _nm = nm;
> lock(_nm);
> }
>
> static void lock(CompiledMethod* method) {
> if (method == NULL) return;
> lock_nmethod(method);
> }
>
> static void unlock(CompiledMethod* method) {
> if (method == NULL) return;
> unlock_nmethod(method);
> }
>
> nmethodLocker() { _nm = NULL; }
> ~nmethodLocker() {
> unlock(_nm);
> }
>
> CompiledMethod* code() { return _nm; }
> void set_code(CompiledMethod* new_nm) {
> unlock(_nm); // note: This works even if _nm==new_nm.
> _nm = new_nm;
> lock(_nm);
> }
> };
>
>
> // QQQ might we make this work from a frame??
> nmethodLocker::nmethodLocker(address pc) {
> CodeBlob* cb = CodeCache::find_blob(pc);
> guarantee(cb != NULL && cb->is_compiled(), "bad pc for a nmethod
> found");
> _nm = cb->as_compiled_method();
> lock_nmethod(_nm);
> }
>
> // Only JvmtiDeferredEvent::compiled_method_unload_event()
> // should pass zombie_ok == true.
> void nmethodLocker::lock_nmethod(CompiledMethod* cm, bool zombie_ok) {
> if (cm == NULL) return;
> if (cm->is_aot()) return; // FIXME: Revisit once _lock_count is
> added to aot_method
> nmethod* nm = cm->as_nmethod();
> Atomic::inc(&nm->_lock_count);
> assert(zombie_ok || !nm->is_zombie(), "cannot lock a zombie method");
> }
>
> void nmethodLocker::unlock_nmethod(CompiledMethod* cm) {
> if (cm == NULL) return;
> if (cm->is_aot()) return; // FIXME: Revisit once _lock_count is
> added to aot_method
> nmethod* nm = cm->as_nmethod();
> Atomic::dec(&nm->_lock_count);
> assert(nm->_lock_count >= 0, "unmatched nmethod lock/unlock");
> }
>
> bool is_locked_by_vm() const { return _lock_count
> >0; }
>
>
>
> The function is_locked_by_vm() must be used to prevent unloading an
> nmethod in use.
> But it is used only in the NMethodSweeper by:
>
> NMethodSweeper::process_compiled_method()
> called from NMethodSweeper::sweep_code_cache()
> called from NMethodSweeper::possibly_sweep()
> called from NMethodSweeper::sweeper_loop()
> called from sweeper_thread_entry()
> called from
> CodeCacheSweeperThread::CodeCacheSweeperThread()
> called from CompileBroker::make_thread()
> called from
> CompileBroker::init_compiler_sweeper_threads()
>
>
> NMethodSweeper::possibly_flush()
> called from NMethodSweeper::process_compiled_method()
>
>
>
> From the other hand, it seems, the nmethod::make_unloaded() cleans up
> the nm->_method value:
>
> // If _method is already NULL the Method* is about to be unloaded,
> // so we don't have to break the cycle. Note that it is possible to
> // have the Method* live here, in case we unload the nmethod because
> // it is pointing to some oop (other than the Method*) being unloaded.
> if (_method != NULL) {
> // OSR methods point to the Method*, but the Method* does not
> // point back!
> if (_method->code() == this) {
> _method->clear_code(); // Break a cycle
> }
> _method = NULL; // Clear the method of this dead
> nmethod <== !!
> }
>
> but it does not care about the nmethod locking api above.
>
>
> The nmethod::make_unloaded() is called from this method:
>
> // If this oop is not live, the nmethod can be unloaded.
> bool nmethod::can_unload(BoolObjectClosure* is_alive, oop* root, bool
> unloading_occurred) {
> assert(root != NULL, "just checking");
> oop obj = *root;
> if (obj == NULL || is_alive->do_object_b(obj)) {
> return false;
> }
>
> // If ScavengeRootsInCode is true, an nmethod might be unloaded
> // simply because one of its constant oops has gone dead.
> // No actual classes need to be unloaded in order for this to occur.
> assert(unloading_occurred || ScavengeRootsInCode, "Inconsistency in
> unloading");
> make_unloaded(is_alive,
> obj); <== !!
> return true;
> }
>
> In its turn the nmethod::make_unloaded() is called from:
>
>
> bool nmethod::unload_if_dead_at(RelocIterator* iter_at_oop,
> BoolObjectClosure *is_alive, bool unloading_occurred) {
> assert(iter_at_oop->type() == relocInfo::oop_type, "Wrong relocation
> type");
>
> oop_Relocation* r = iter_at_oop->oop_reloc();
> // Traverse those oops directly embedded in the code.
> // Other oops (oop_index>0) are seen as part of scopes_oops.
> assert(1 == (r->oop_is_immediate()) +
> (r->oop_addr() >= oops_begin() && r->oop_addr() < oops_end()),
> "oop must be found in exactly one place");
> if (r->oop_is_immediate() && r->oop_value() != NULL) {
> // Unload this nmethod if the oop is dead.
> if (can_unload(is_alive, r->oop_addr(), unloading_occurred)) {
> return true;;
> }
> }
> return false;
> }
>
> bool nmethod::do_unloading_scopes(BoolObjectClosure* is_alive, bool
> unloading_occurred) {
> // Scopes
> for (oop* p = oops_begin(); p < oops_end(); p++) {
> if (*p == Universe::non_oop_word()) continue; // skip non-oops
> if (can_unload(is_alive, p, unloading_occurred)) {
> return true;
> }
> }
> return false;
> }
>
> #if INCLUDE_JVMCI
> bool nmethod::do_unloading_jvmci(BoolObjectClosure* is_alive, bool
> unloading_occurred) {
> bool is_unloaded = false;
> // Follow JVMCI method
> BarrierSet* bs = Universe::heap()->barrier_set();
> if (_jvmci_installed_code != NULL) {
> if (_jvmci_installed_code->is_a(HotSpotNmethod::klass()) &&
> HotSpotNmethod::isDefault(_jvmci_installed_code)) {
> if (!is_alive->do_object_b(_jvmci_installed_code)) {
> clear_jvmci_installed_code();
> }
> } else {
> if (can_unload(is_alive, (oop*)&_jvmci_installed_code,
> unloading_occurred)) {
> return true;
> }
> }
> }
>
> if (_speculation_log != NULL) {
> if (!is_alive->do_object_b(_speculation_log)) {
> bs->write_ref_nmethod_pre(&_speculation_log, this);
> _speculation_log = NULL;
> bs->write_ref_nmethod_post(&_speculation_log, this);
> }
> }
> return is_unloaded;
> }
> #endif
>
>
> The only caller of the the unload_if_dead_at() and do_unloading_scopes()
> is nmethod::do_unloading_oops():
>
> bool nmethod::do_unloading_oops(address low_boundary, BoolObjectClosure*
> is_alive, bool unloading_occurred) {
> // Compiled code
> {
> RelocIterator iter(this, low_boundary);
> while (iter.next()) {
> if (iter.type() == relocInfo::oop_type) {
> if (unload_if_dead_at(&iter, is_alive, unloading_occurred)) {
> return true;
> }
> }
> }
> }
> return do_unloading_scopes(is_alive, unloading_occurred);
> }
>
> Let's skip AOT part.
>
> Both
> nmethod::do_unloading_oops() and
> nmethod::do_unloading_jvmci()
> called from CompiledMethod::do_unloading()
> and CompiledMethod::do_unloading_parallel().
>
>
> CompiledMethod::do_unloading_parallel()
> called from G1CodeCacheUnloadingTask::clean_nmethod()
> called from G1CodeCacheUnloadingTask::work_first_pass()
> and G1CodeCacheUnloadingTask::work_second_pass()
>
> called from G1ParallelCleaningTask::G1ParallelCleaningTask()
> called from G1CollectedHeap::parallel_cleaning()
> called from
> G1ConcurrentMark::weakRefsWorkParallelPart()
> called from G1ConcurrentMark::weakRefsWork()
> called from
> G1ConcurrentMark::checkpointRootsFinal()
> . . .
>
> Now, I'm lost...
> It is hard to find out how all these sub-call-tree should transitively
> call the NMethodSweeper::process_compiled_method() function which uses
> the function is_locked_by_vm() to synchronize with
> post_compiled_method_load().
>
> Just some initial (very stupid) analysis... :-)
>
>
> Thanks,
> Serguei
>
>
>
>
>
> On 5/9/18 01:02, serguei.spitsyn at oracle.com wrote:
> > Hi Ioi and David,
> >
> > Ioi, thank you for the disassembly.
> > It helps for sure!
> >
> >
> > Below are 3 call sites for the
> > nmethod::post_compiled_method_load_event():
> >
> > void ciEnv::register_method(ciMethod* target, . . .) {
> > . . .
> > if (nm != NULL) {
> > // JVMTI -- compiled method notification (must be done outside lock)
> > nm->post_compiled_method_load_event(); <== !!
> > } else {
> > . . .
> > }
> >
> > }
> >
> > JVMCIEnv::CodeInstallResult JVMCIEnv::register_method(. . .) {
> > . . .
> > // JVMTI -- compiled method notification (must be done outside lock)
> > if (nm != NULL) {
> > nm->post_compiled_method_load_event(); <== !!
> > . . .
> > }
> > return result;
> > }
> >
> > void AdapterHandlerLibrary::create_native_wrapper(const methodHandle&
> > method) {
> > . . .
> > if (nm != NULL) {
> > . . .
> > nm->post_compiled_method_load_event(); <== !!
> > }
> > }
> >
> > The nmethod::post_compiled_method_load_event() creates a
> > JvmtiDeferredEvent
> > compiled_load_event entry and enqueues it for the ServiceThread
> > processing.
> >
> > void nmethod::post_compiled_method_load_event() {
> >
> > Method* moop = method();
> > HOTSPOT_COMPILED_METHOD_LOAD(
> > (char *) moop->klass_name()->bytes(),
> > moop->klass_name()->utf8_length(),
> > (char *) moop->name()->bytes(),
> > moop->name()->utf8_length(),
> > (char *) moop->signature()->bytes(),
> > moop->signature()->utf8_length(),
> > insts_begin(), insts_size());
> >
> > if (JvmtiExport::should_post_compiled_method_load() ||
> > JvmtiExport::should_post_compiled_method_unload()) {
> > get_and_cache_jmethod_id();
> > }
> >
> > if (JvmtiExport::should_post_compiled_method_load()) {
> > // Let the Service thread (which is a real Java thread) post the
> > event
> > MutexLockerEx ml(Service_lock, Mutex::_no_safepoint_check_flag);
> > JvmtiDeferredEventQueue::enqueue(
> > JvmtiDeferredEvent::compiled_method_load_event(this)); <== !!
> > }
> > }
> >
> >
> > The vmtiDeferredEvent::compiled_method_load_event() below uses
> > nmethodLocker::lock_nmethod(nm) to protect the nmethod* nm
> > from being updated:
> >
> > JvmtiDeferredEvent JvmtiDeferredEvent::compiled_method_load_event(
> > nmethod* nm) {
> > JvmtiDeferredEvent event =
> > JvmtiDeferredEvent(TYPE_COMPILED_METHOD_LOAD);
> > event._event_data.compiled_method_load =
> > nm; <== !!
> > // Keep the nmethod alive until the ServiceThread can process
> > // this deferred event.
> > nmethodLocker::lock_nmethod(nm); <== !!
> > return event;
> > }
> >
> > void JvmtiDeferredEventQueue::enqueue(const JvmtiDeferredEvent& event) {
> > assert(Service_lock->owned_by_self(), "Must own Service_lock");
> >
> > process_pending_events();
> >
> > // Events get added to the end of the queue (and are pulled off the
> > front).
> > QueueNode* node = new QueueNode(event);
> > if (_queue_tail == NULL) {
> > _queue_tail = _queue_head = node;
> > } else {
> > assert(_queue_tail->next() == NULL, "Must be the last element in
> > the list");
> > _queue_tail->set_next(node);
> > _queue_tail = node;
> > }
> >
> > Service_lock->notify_all();
> > assert((_queue_head == NULL) == (_queue_tail == NULL),
> > "Inconsistent queue markers");
> > }
> >
> > JvmtiDeferredEvent JvmtiDeferredEventQueue::dequeue() {
> > assert(Service_lock->owned_by_self(), "Must own Service_lock");
> >
> > process_pending_events();
> >
> > assert(_queue_head != NULL, "Nothing to dequeue");
> >
> > if (_queue_head == NULL) {
> > // Just in case this happens in product; it shouldn't but let's
> > not crash
> > return JvmtiDeferredEvent();
> > }
> >
> > QueueNode* node = _queue_head;
> > _queue_head = _queue_head->next();
> > if (_queue_head == NULL) {
> > _queue_tail = NULL;
> > }
> >
> > assert((_queue_head == NULL) == (_queue_tail == NULL),
> > "Inconsistent queue markers");
> >
> > JvmtiDeferredEvent event = node->event();
> > delete node;
> > return event;
> > }
> >
> > The ServiceThread (different thread which is a JavaThread)
> > takes the event from the JvmtiDeferredEventQueue queue
> > and posts it.
> >
> > void ServiceThread::service_thread_entry(JavaThread* jt, TRAPS) {
> > . . .
> > if (has_jvmti_events) {
> > jvmti_event = JvmtiDeferredEventQueue::dequeue();
> > <== !!
> > }
> > }
> >
> > if (has_jvmti_events) {
> > jvmti_event.post(); <== !!
> > }
> > . . .
> > }
> >
> > The JvmtiDeferredEvent::post() below makes the call:
> > JvmtiExport::post_compiled_method_load(nm);
> >
> > and only then unlocks the nmethod nm with
> > nmethodLocker::unlock_nmethod(nm).
> >
> >
> > void JvmtiDeferredEvent::post() {
> > assert(ServiceThread::is_service_thread(Thread::current()),
> > "Service thread must post enqueued events");
> > switch(_type) {
> > case TYPE_COMPILED_METHOD_LOAD: {
> > nmethod* nm = _event_data.compiled_method_load;
> > JvmtiExport::post_compiled_method_load(nm); <== !!
> > // done with the deferred event so unlock the nmethod
> > nmethodLocker::unlock_nmethod(nm); <== !!
> > break;
> > }
> > . . .
> > default:
> > ShouldNotReachHere();
> > }
> > }
> >
> > The printed disassembly shows where we observe the crash in the
> > functions below:
> >
> > void JvmtiExport::post_compiled_method_load(nmethod *nm) {
> > if (JvmtiEnv::get_phase() < JVMTI_PHASE_PRIMORDIAL) {
> > return;
> > }
> > JavaThread* thread = JavaThread::current();
> >
> > EVT_TRIG_TRACE(JVMTI_EVENT_COMPILED_METHOD_LOAD,
> > ("[%s] method compile load event triggered",
> > JvmtiTrace::safe_get_thread_name(thread)));
> >
> > JvmtiEnvIterator it;
> > for (JvmtiEnv* env = it.first(); env != NULL; env = it.next(env)) {
> > if (env->is_enabled(JVMTI_EVENT_COMPILED_METHOD_LOAD)) {
> > if (env->phase() == JVMTI_PHASE_PRIMORDIAL) {
> > continue;
> > }
> > EVT_TRACE(JVMTI_EVENT_COMPILED_METHOD_LOAD,
> > ("[%s] class compile method load event sent %s.%s ",
> > JvmtiTrace::safe_get_thread_name(thread),
> > (nm->method() == NULL) ? "NULL" :
> > nm->method()->klass_name()->as_C_string(),
> > (nm->method() == NULL) ? "NULL" :
> > nm->method()->name()->as_C_string()));
> > ResourceMark rm(thread);
> > HandleMark hm(thread);
> >
> > // Add inlining information
> > jvmtiCompiledMethodLoadInlineRecord* inlinerecord =
> > create_inline_record(nm);
> > // Pass inlining information through the void pointer
> > JvmtiCompiledMethodLoadEventMark jem(thread, nm,
> > inlinerecord); <== !!
> > JvmtiJavaThreadEventTransition jet(thread);
> > jvmtiEventCompiledMethodLoad callback =
> > env->callbacks()->CompiledMethodLoad;
> > if (callback != NULL) {
> > (*callback)(env->jvmti_external(), jem.jni_methodID(),
> > jem.code_size(), jem.code_data(), jem.map_length(),
> > jem.map(), jem.compile_info());
> > }
> > }
> > }
> > }
> >
> > class JvmtiCompiledMethodLoadEventMark : public JvmtiMethodEventMark {
> > . . .
> > public:
> > JvmtiCompiledMethodLoadEventMark(JavaThread *thread, nmethod *nm,
> > void* compile_info_ptr = NULL)
> > : JvmtiMethodEventMark(thread,methodHandle(thread,
> > nm->method())) { <== !!
> > . . .
> > }
> > . . .
> > }
> >
> > class JvmtiMethodEventMark : public JvmtiThreadEventMark {
> > . . .
> > JvmtiMethodEventMark(JavaThread *thread, methodHandle method) :
> > JvmtiThreadEventMark(thread),
> > _mid(to_jmethodID(method))
> > {}; <== line 234 !!
> > jmethodID jni_methodID() { return _mid; }
> > };
> >
> >
> > All the fragments above show the code path correctly operates with
> > nmethod
> > by protecting it with the nmethodLocker::lock_nmethod(nm) until the event
> > has been posted.
> >
> > There can be several guesses about the above:
> > - the above is correct but some other code abuses the convention and
> > updates the nmethod without grabbing the nmethod lock
> > - the above operations with the nmethod is incorrect (miss some steps)
> > - some other reason (I doubt it is the case)
> >
> >
> > One conclusion is that it is unlikely an issue in the JVMTI agent as I
> > suspected initially.
> > It would still be valuable to have more details about a scenario that
> > reproduced this issue.
> > It is probably enough information to file a bug.
> >
> > Any opinions on the above?
> > Any ideas about what can be broken?
> >
> > Thanks,
> > Serguei
> >
> >
> > On 5/8/18 23:00, David Holmes wrote:
> >> On 9/05/2018 3:06 PM, Ioi Lam wrote:
> >>> Serguei & Nezih,
> >>>
> >>> I've looked at the crash log and it seems like nm->method() is NULL,
> >>> which caused the crash.
> >>>
> >>> I am not familiar with the JVMTI internals or nmethods, so I am just
> >>> guessing here -- is it legal for nm->method() to be NULL? Or, is it
> >>> possible that someone has cleared nm->method() before it's processed
> >>> by post_compiled_method_load?
> >>
> >> I was wondering the same thing. Vladimir K. pointed me at:
> >>
> >>
> http://hg.openjdk.java.net/jdk/jdk/file/ae0ebd3cf949/src/hotspot/share/runtime/sweeper.hpp#l42
> >>
> >>
> >> for the nmethod lifecycle. Given this is actually a deferred event, I
> >> too wonder if the nmethod may have been made not-entrant or even
> >> zombie, since the event was posted.
> >>
> >> David
> >> -----
> >>
> >>> Details from the crash log:
> >>>
> >>> =================================
> >>> Native frames: (J=compiled Java code, A=aot compiled Java code,
> >>> j=interpreted, Vv=VM code, C=native code)
> >>> V [libjvm.so+0x99e365]
> >>> JvmtiExport::post_compiled_method_load(nmethod*)+0x275
> >>> V [libjvm.so+0x9a6614] JvmtiDeferredEvent::post()+0x44
> >>> V [libjvm.so+0xc2c507]
> >>> ServiceThread::service_thread_entry(JavaThread*, Thread*)+0x337
> >>> V [libjvm.so+0xceee58] JavaThread::thread_main_inner()+0xd8
> >>> V [libjvm.so+0xb6de12] thread_native_entry(Thread*)+0xf2
> >>> C [libpthread.so.0+0x7e25] start_thread+0xc5
> >>> ...
> >>> R14=0x0000000000000000 is an unknown value
> >>> =================================
> >>>
> >>> Here's a disassembly of JvmtiExport::post_compiled_method_load from
> >>> JDK 9.0.1
> >>>
> >>>
> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/b756e7a2ec33/src/share/vm/prims/jvmtiExport.cpp
> >>>
> >>>
> >>> 2051 HandleMark hm(thread);
> >>> 0x00007ffff67dd239 <+329>: mov %rax,-0x140(%rbp)
> >>> 0x00007ffff67dd240 <+336>: callq 0x7ffff66149d0
> >>> <HandleMark::initialize(Thread*)>
> >>>
> >>> 2054 jvmtiCompiledMethodLoadInlineRecord* inlinerecord =
> >>> create_inline_record(nm);
> >>> 0x00007ffff67dd245 <+341>: mov %r12,%rdi
> >>> 0x00007ffff67dd248 <+344>: callq 0x7ffff67d4090
> >>> <create_inline_record(nmethod*)>
> >>>
> >>> 2055 // Pass inlining information through the void pointer
> >>> 2056 JvmtiCompiledMethodLoadEventMark jem(thread, nm,
> >>> inlinerecord);
> >>> 0x00007ffff67dd24d <+349>: mov %rax,-0x1a8(%rbp)
> >>> 0x00007ffff67dd254 <+356>: mov 0x70(%r12),%rax
> >>> 0x00007ffff67dd259 <+361>: mov %rbx,-0x178(%rbp)
> >>> 0x00007ffff67dd260 <+368>: test %rax,%rax
> >>> 0x00007ffff67dd263 <+371>: mov %rax,-0x180(%rbp)
> >>> 0x00007ffff67dd26a <+378>: je 0x7ffff67dd282
> >>> <JvmtiExport::post_compiled_method_load(nmethod*)+402>
> >>> 0x00007ffff67dd26c <+380>: mov %rax,-0xa0(%rbp)
> >>> 0x00007ffff67dd273 <+387>: mov 0x160(%rbx),%rdi
> >>> 0x00007ffff67dd27a <+394>: mov %r13,%rsi
> >>> 0x00007ffff67dd27d <+397>: callq 0x7ffff671d5f0
> >>> <GrowableArray<Metadata*>::append(Metadata* const&)>
> >>> 0x00007ffff67dd282 <+402>: lea 0x218(%rbx),%rax
> >>> 0x00007ffff67dd289 <+409>: mov %rbx,-0xf0(%rbp)
> >>> 0x00007ffff67dd290 <+416>: movl $0x0,-0xe0(%rbp)
> >>> 0x00007ffff67dd29a <+426>: mov %rax,-0xe8(%rbp)
> >>> 0x00007ffff67dd2a1 <+433>: mov 0x3c8(%rbx),%rax
> >>> 0x00007ffff67dd2a8 <+440>: test %rax,%rax
> >>> 0x00007ffff67dd2ab <+443>: je 0x7ffff67dd2b6
> >>> <JvmtiExport::post_compiled_method_load(nmethod*)+454>
> >>> 0x00007ffff67dd2ad <+445>: mov 0x10(%rax),%eax
> >>> 0x00007ffff67dd2b0 <+448>: mov %eax,-0xe0(%rbp)
> >>> 0x00007ffff67dd2b6 <+454>: mov 0x38(%rbx),%r14
> >>> 0x00007ffff67dd2ba <+458>: mov %rbx,%rdi
> >>> 0x00007ffff67dd2bd <+461>: callq 0x7ffff6707d50
> >>> <JNIHandleBlock::allocate_block(Thread*)>
> >>> 0x00007ffff67dd2c2 <+466>: lea 0x1f8(%rbx),%rdi
> >>> 0x00007ffff67dd2c9 <+473>: mov %rbx,%rsi
> >>> 0x00007ffff67dd2cc <+476>: mov %r14,0x118(%rax)
> >>> 0x00007ffff67dd2d3 <+483>: mov %rax,0x38(%rbx)
> >>> 0x00007ffff67dd2d7 <+487>: callq 0x7ffff6559410
> >>> <JavaFrameAnchor::make_walkable(JavaThread*)>
> >>> 0x00007ffff67dd2dc <+492>: mov 0x1f0(%rbx),%rsi
> >>> 0x00007ffff67dd2e3 <+499>: mov -0xf0(%rbp),%rdi
> >>> 0x00007ffff67dd2ea <+506>: callq 0x7ffff6708520
> >>> <JNIHandles::make_local(Thread*, oopDesc*)>
> >>> 0x00007ffff67dd2ef <+511>: mov -0x1a0(%rbp),%rsi
> >>> 0x00007ffff67dd2f6 <+518>: mov %rax,-0xd8(%rbp)
> >>> 0x00007ffff67dd2fd <+525>: lea -0x170(%rbp),%rax
> >>> 0x00007ffff67dd304 <+532>: mov %rax,%rdi
> >>> 0x00007ffff67dd307 <+535>: mov %rax,-0x1b0(%rbp)
> >>>
> >>>
> >>> // JvmtiMethodEventMark(JavaThread *thread, methodHandle method) :
> >>> // JvmtiThreadEventMark(thread),
> >>> 234 _mid(to_jmethodID(method)) {}; <-- this line
> >>>
> >>> 0x00007ffff67dd30e <+542>: callq 0x7ffff66140a0
> >>> <methodHandle::methodHandle(methodHandle const&)>
> >>> 0x00007ffff67dd313 <+547>: mov -0x170(%rbp),%r14
> >>> 0x00007ffff67dd31a <+554>: movq $0x0,-0x98(%rbp)
> >>> 0x00007ffff67dd325 <+565>: test %r14,%r14
> >>> 0x00007ffff67dd328 <+568>: mov %r14,-0xa0(%rbp)
> >>> 0x00007ffff67dd32f <+575>: je 0x7ffff67dd365
> >>> <JvmtiExport::post_compiled_method_load(nmethod*)+629>
> >>>
> >>> 0x00007ffff67dd331 <+577>: data16 lea
> >>> 0x946ca7(%rip),%rdi # 0x7ffff7123fe0
> >>> 0x00007ffff67dd339 <+585>: data16 data16 callq 0x7ffff60b11d0
> >>> <__tls_get_addr at plt>
> >>> 0x00007ffff67dd341 <+593>: lea -0x188(%rbp),%rsi
> >>> 0x00007ffff67dd348 <+600>: mov %r14,-0x188(%rbp)
> >>> 0x00007ffff67dd34f <+607>: mov (%rax),%rax
> >>> 0x00007ffff67dd352 <+610>: mov %rax,-0x98(%rbp)
> >>> 0x00007ffff67dd359 <+617>: mov 0x160(%rax),%rdi
> >>> 0x00007ffff67dd360 <+624>: callq 0x7ffff671d5f0
> >>> <GrowableArray<Metadata*>::append(Metadata* const&)>
> >>>
> >>> >>> CRASH %r14 == 0x0
> >>> 198 jmethodID to_jmethodID(methodHandle method) { return
> >>> method->jmethod_id(); }
> >>>
> >>> 0x00007ffff67dd365 <+629>: mov 0x8(%r14),%rax
> >>>
> >>>
> >>> Hope this helps
> >>> - Ioi
> >>>
> >>> On 5/8/18 4:51 PM, serguei.spitsyn at oracle.com wrote:
> >>>> Hi Nezih,
> >>>>
> >>>> Crashing in the JvmtiExport::post_compiled_method_load function
> >>>> does not prove itself that it is a VM issue.
> >>>> And there are mo specific details in the log that help to
> >>>> understand this.
> >>>> Filing a bug with this log is almost useless as it would not help
> >>>> to get to the root cause.
> >>>> Also, there is still a pretty big chance the real problem is in the
> >>>> agent.
> >>>>
> >>>> My advice is to get as more details about this crash as possible.
> >>>> The best you can do is to provide a stand alone test case.
> >>>> Even if the issue is in the agent it will help to figure it out.
> >>>>
> >>>> Thanks,
> >>>> Serguei
> >>>>
> >>>>
> >>>> On 5/8/18 16:38, nezih yigitbasi wrote:
> >>>>> Hi Serguei,
> >>>>> We don't have a repro unfortunately. We have seen this only once
> >>>>> so far in our production environment.
> >>>>>
> >>>>> Do you think this can be an agent issue? I thought like if it was
> >>>>> an agent issue the stack trace would show it. The stack trace,
> >>>>> however, ends in the VM code -- the crash happens in the
> >>>>> "JvmtiExport::post_compiled_method_load" method.
> >>>>>
> >>>>> Thanks,
> >>>>> Nezih
> >>>>>
> >>>>> 2018-05-08 16:34 GMT-07:00 serguei.spitsyn at oracle.com
> >>>>> <mailto:serguei.spitsyn at oracle.com> <serguei.spitsyn at oracle.com
> >>>>> <mailto:serguei.spitsyn at oracle.com>>:
> >>>>>
> >>>>> Hi Nezih,
> >>>>>
> >>>>> You error file with log does not help much.
> >>>>> Could you, please, reproduce this issue with the debug or
> >>>>> fastdebug build?
> >>>>> Also, a standalone test case is better to provide to
> >>>>> increase chances the issue to be investigated and fixed.
> >>>>>
> >>>>> In general, a JVMTI agent has to be written very carefully as it
> >>>>> can easily cause the VM to crash.
> >>>>> So, it makes sense to look at the agent implementation first to
> >>>>> make sure it does not do anything wrong.
> >>>>> It is what we normally do first and why a test case that
> >>>>> demonstrates the problem is needed.
> >>>>>
> >>>>> Thanks,
> >>>>> Serguei
> >>>>>
> >>>>>
> >>>>>
> >>>>> On 5/8/18 16:00, nezih yigitbasi wrote:
> >>>>>> Hi,
> >>>>>>
> >>>>>> First of all, sorry for bringing up a crash issue in this dev
> >>>>>> mailing list, but the crash report submission page
> >>>>>> (https://bugreport.java.com/bugreport/crash.jsp
> >>>>>> <https://bugreport.java.com/bugreport/crash.jsp>) doesn't list
> >>>>>> Java 9 in the release drop down, so I couldn't report it there.
> >>>>>>
> >>>>>> We recently got a crash with Java 9.0.1+11 with an interesting
> >>>>>> stack ending at "JvmtiExport::post_compiled_method_load()"
> >>>>>> (entire error file is here
> >>>>>> <
> https://gist.github.com/nezihyigitbasi/52a58698cc9acfcab21c69d00bd0cef2>).
>
> >>>>>>
> >>>>>> A google search didn't end up with much info, so I just wanted
> >>>>>> to check with this mailing list to see whether anyone has any
> >>>>>> ideas to investigate this further.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Nezih
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20180509/0c1ebb79/attachment-0001.html>
More information about the serviceability-dev
mailing list