how to bring jvm to safepoint
Tony Guan
guanxiaohua at gmail.com
Tue Aug 3 16:58:26 PDT 2010
Hi Tom,
Thanks for the reminder, I will keep that in mind.
I will call GC at allocation time anyway (without modifying
dtrace_method_exit) for I think I have a better idea for my current
project.
Thanks a lot, everyone!
Tony (Xiaohua Guan)
On Tue, Aug 3, 2010 at 6:51 PM, Tom Rodriguez <tom.rodriguez at oracle.com> wrote:
> I suspect this isn't going work properly with compiled code. In general you can't simply change JRT_LEAF to JRT_ENTRY because JRT_ENTRY functions must be called from locations that include proper debug info and we don't emit any for leaf calls. It will likely work for the interpreter since the stack is simpler to walk but I expect you will have problems of some sort. I guess you add debug info to those calls in compiled code but it's not obvious to me where you would get that from.
>
> tom
>
> On Aug 3, 2010, at 4:38 PM, Tony Guan wrote:
>
>> Hi Everyone,
>>
>> Please ignore my previous message, I just forgot to update the jvm for testing.
>>
>> Actually, changing dtrace_method_exit to be JRT_ENTRY solved the
>> problem! It works out.
>>
>> Thank you, David!
>>
>>
>> Cheers!
>>
>> Tony (Xiaohua Guan)
>>
>>
>>
>> On Tue, Aug 3, 2010 at 6:20 PM, Tony Guan <guanxiaohua at gmail.com> wrote:
>>> Hi David,
>>>
>>> I changed dtrace_method_exit to JRT_ENTRY from JRT_LEAF, and still
>>> stopped by the same failure. Does it have anything to do with the
>>> method sun.misc.Unsafe.park()?
>>>
>>> Thanks!
>>>
>>> Tony (Xiaohua Guan)
>>>
>>>
>>>
>>> On Sun, Aug 1, 2010 at 3:50 PM, Tony Guan <guanxiaohua at gmail.com> wrote:
>>>> Hi David,
>>>>
>>>> Thanks for the explanation, now I get to understand it.
>>>>
>>>> Have to say, it's a nice failure for me. :)
>>>>
>>>> Tony (Xiaohua Guan)
>>>>
>>>>
>>>>
>>>> On Sat, Jul 31, 2010 at 10:43 PM, David Holmes <David.Holmes at oracle.com> wrote:
>>>>> Hi Tony,
>>>>>
>>>>> Now it is all clear :)
>>>>>
>>>>>> It's exactly how the error occurred, I am monitoring the
>>>>>> sun.misc.Unsafe.park() method call with dtrace(no dtrace agent, just
>>>>>> intercepting inside jvm implementation), once this method gets called,
>>>>>> I will call GenCollectedHeap::invoke_transEden_collection() to start a
>>>>>> GC using VMThread::Execute(&op).
>>>>>
>>>>> Right, you are using:
>>>>>
>>>>> _ZN13SharedRuntime18dtrace_method_exitEP10JavaThreadP13methodOopDesc+0x255
>>>>>
>>>>> but dtrace_method_exit is defined as a runtime "leaf" method (JRT_LEAF).
>>>>> "leaf" methods are not allowed to perform certain actions and in debug mode
>>>>> we have JRT_Leaf_Verifier that is used to check those actions. A
>>>>> JRT_Leaf_Verifier is a No_Safepoint_Verifier - hence as I said before you
>>>>> get the assertion failure due to this.
>>>>>
>>>>> When you do the VMThread::execute to initiate the safepoint operation for
>>>>> your GC action, the current thread will block waiting for it to complete.
>>>>> Part of that blocking involves a safepoint check and hence the system finds
>>>>> that the thread is attempting to enter a safepoint from a "leaf" method -
>>>>> and that is not allowed.
>>>>>
>>>>> You might just try changing the definition of dtrace_method_exit to be
>>>>> JRT_ENTRY rather than JRT_LEAF, but I don't know if that will have
>>>>> unexpected side-effects.
>>>>>
>>>>> There are lots of rules (not necessarily clearly documented) that control
>>>>> how you can call into the VM runtime and how different parts of the VM
>>>>> runtime can call other parts of the runtime.
>>>>>
>>>>> Cheers,
>>>>> David Holmes
>>>>>
>>>>> Tony Guan said the following on 08/01/10 08:10:
>>>>>>
>>>>>> Hi David & Tom,
>>>>>>
>>>>>> Please see the decoded stack below(also full decoded log is attached):
>>>>>>
>>>>>> "
>>>>>> Current thread (0x00000000012ed000): JavaThread "RMI TCP
>>>>>> Connection(idle)" daemon [_thread_in_Java, id=30929,
>>>>>> stack(0x000000004073b000,0x000000004083c000)]
>>>>>>
>>>>>> Stack: [0x000000004073b000,0x000000004083c000],
>>>>>> sp=0x0000000040839d60, free space=3fb0000000000000000k
>>>>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
>>>>>> code)
>>>>>> V [libjvm.so+0x8cad18];; _ZN7VMError6reportEP12outputStream+0xb72
>>>>>> V [libjvm.so+0x8cbe38];; _ZN7VMError14report_and_dieEv+0x5f6
>>>>>> V [libjvm.so+0x4039f1];; _Z12report_fatalPKciS0_+0x6b
>>>>>> V [libjvm.so+0x87a183];;
>>>>>> _ZN6Thread31check_for_valid_safepoint_stateEb+0x33
>>>>>> V [libjvm.so+0x8e1127];; _ZN8VMThread7executeEP12VM_Operation+0x49
>>>>>> V [libjvm.so+0x4d52ab];;
>>>>>> _ZN16GenCollectedHeap27invoke_transEden_collectionEi+0x33
>>>>>> V [libjvm.so+0x7ee63d];;
>>>>>> _ZN13SharedRuntime18dtrace_method_exitEP10JavaThreadP13methodOopDesc+0x255
>>>>>> j sun.misc.Unsafe.park(ZJ)V+0
>>>>>> j
>>>>>> java.util.concurrent.locks.LockSupport.parkNanos(Ljava/lang/Object;J)V+20
>>>>>> j
>>>>>> java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Ljava/util/concurrent/SynchronousQueue$TransferStack$SNode;ZJ)Ljava/util/concurrent/SynchronousQueue$TransferStack$SNode;+174
>>>>>> j
>>>>>> java.util.concurrent.SynchronousQueue$TransferStack.transfer(Ljava/lang/Object;ZJ)Ljava/lang/Object;+102
>>>>>> j
>>>>>> java.util.concurrent.SynchronousQueue.poll(JLjava/util/concurrent/TimeUnit;)Ljava/lang/Object;+11
>>>>>> j
>>>>>> java.util.concurrent.ThreadPoolExecutor.getTask()Ljava/lang/Runnable;+141
>>>>>> j
>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+17
>>>>>> j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
>>>>>> j java.lang.Thread.run()V+11
>>>>>> v ~StubRoutines::call_stub
>>>>>> V [libjvm.so+0x555ecc];;
>>>>>>
>>>>>> _ZN9JavaCalls11call_helperEP9JavaValueP12methodHandleP17JavaCallArgumentsP6Thread+0x474
>>>>>> V [libjvm.so+0x747a34];;
>>>>>>
>>>>>> _ZN2os20os_exception_wrapperEPFvP9JavaValueP12methodHandleP17JavaCallArgumentsP6ThreadES1_S3_S5_S7_+0x32
>>>>>> V [libjvm.so+0x554b55];;
>>>>>>
>>>>>> _ZN9JavaCalls4callEP9JavaValue12methodHandleP17JavaCallArgumentsP6Thread+0x6f
>>>>>> V [libjvm.so+0x5551f0];;
>>>>>>
>>>>>> _ZN9JavaCalls12call_virtualEP9JavaValue11KlassHandle12symbolHandleS3_P17JavaCallArgumentsP6Thread+0x158
>>>>>> V [libjvm.so+0x5553cc];;
>>>>>>
>>>>>> _ZN9JavaCalls12call_virtualEP9JavaValue6Handle11KlassHandle12symbolHandleS4_P6Thread+0x86
>>>>>> V [libjvm.so+0x5be6cd];; _Z12thread_entryP10JavaThreadP6Thread+0x9f
>>>>>> V [libjvm.so+0x879869];; _ZN10JavaThread17thread_main_innerEv+0xc9
>>>>>> V [libjvm.so+0x87c1b5];; _ZN10JavaThread3runEv+0x2c1
>>>>>> V [libjvm.so+0x750455];; _Z10java_startP6Thread+0x16f
>>>>>> "
>>>>>>
>>>>>> It's exactly how the error occurred, I am monitoring the
>>>>>> sun.misc.Unsafe.park() method call with dtrace(no dtrace agent, just
>>>>>> intercepting inside jvm implementation), once this method gets called,
>>>>>> I will call GenCollectedHeap::invoke_transEden_collection() to start a
>>>>>> GC using VMThread::Execute(&op).
>>>>>>
>>>>>> At the beginning of VMTHread::Execute(),
>>>>>> check_for_valid_safepoint_state is called like this:
>>>>>>
>>>>>> void VMThread::execute(VM_Operation* op) {
>>>>>> Thread* t = Thread::current();
>>>>>> if (!t->is_VM_thread()) {
>>>>>> // JavaThread or WatcherThread
>>>>>> t->check_for_valid_safepoint_state(true); //And we failed here!!
>>>>>> // New request from Java thread, evaluate prologue
>>>>>> if (!op->doit_prologue()) {
>>>>>> return; // op was cancelled
>>>>>> }
>>>>>> ...
>>>>>> }
>>>>>>
>>>>>> That's why I said the current thread is not a VM_thread. I think I can
>>>>>> make a workaround by invoking the GC at the next allocation request,
>>>>>> which should be in the safepoint. But would be great if I can learn
>>>>>> more about hotspot solving this problem.
>>>>>>
>>>>>> Thanks a lot!
>>>>>>
>>>>>> Tony (Xiaohua Guan)
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sat, Jul 31, 2010 at 12:56 AM, David Holmes <David.Holmes at oracle.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Tom Rodriguez said the following on 07/31/10 05:01:
>>>>>>>>
>>>>>>>> Actually I think the problem is that the thread is in thread_in_Java
>>>>>>>> state. I don't believe you can safepoint from that state, only
>>>>>>>> thread_in_vm.
>>>>>>>
>>>>>>> The reported failure:
>>>>>>>
>>>>>>> # Error: Possible safepoint reached by thread that does not allow it
>>>>>>>
>>>>>>> is here:
>>>>>>>
>>>>>>> void Thread::check_for_valid_safepoint_state(bool potential_vm_operation)
>>>>>>> {
>>>>>>> // Check if current thread is allowed to block at a safepoint
>>>>>>> if (!(_allow_safepoint_count == 0))
>>>>>>> fatal("Possible safepoint reached by thread that does not allow it");
>>>>>>>
>>>>>>> as far as I can see the only code that touches _allow_safepoint_count are
>>>>>>> the No_Safepoint_Verifier classes.
>>>>>>>
>>>>>>> Also the stack shows:
>>>>>>>
>>>>>>> V [libjvm.so+0x7ee63d]
>>>>>>> j sun.misc.Unsafe.park(ZJ)V+0
>>>>>>>
>>>>>>> and so we should have transitioned to _thread_in_vm here.
>>>>>>>
>>>>>>> That said, Tony also stated:
>>>>>>>
>>>>>>>> The reason for the fail is that
>>>>>>>> Thread::check_for_valid_safepoint_state() didn't pass because
>>>>>>>> the currentThread is not a VMThread.
>>>>>>>
>>>>>>> But that is not supported by the error produced.
>>>>>>>
>>>>>>> Tony: why did you make the above comment?
>>>>>>>
>>>>>>> Either way we need to see a decoded stack to get an idea of what is going
>>>>>>> wrong.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> David
>>>>>>>
>>>>>>>> tom
>>>>>>>>
>>>>>>>> On Jul 30, 2010, at 12:04 AM, David Holmes <David.Holmes at oracle.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Sorry Tony I keep forgetting that I don't have a way to decode a
>>>>>>>>> crash log from a VM that you built yourself. :( However you may be
>>>>>>>>> able to decode it yourself please see:
>>>>>>>>>
>>>>>>>>> http://blogs.sun.com/dave/entry/a_tool_to_decode_hs
>>>>>>>>>
>>>>>>>>> for a perl script.
>>>>>>>>>
>>>>>>>>> That said looking at the actual assertion failure I am guessing
>>>>>>>>> that you initiated the VM operation from code where a
>>>>>>>>> No_Safepoint_verifier is active.
>>>>>>>>>
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>> Tony Guan said the following on 07/30/10 15:58:
>>>>>>>>>>
>>>>>>>>>> Hi David, Thanks a lot! I am copying the content to below, and
>>>>>>>>>> for convenience, the full log file is attached. cat
>>>>>>>>>> hs_err_pid30894.log # # A fatal error has been detected by the
>>>>>>>>>> Java Runtime Environment: # # Internal Error
>>>>>>>>>>
>>>>>>>>>> (/home/tony/software/OpenJDK/jdk7/hotspot/src/share/vm/runtime/thread.cpp:777),
>>>>>>>>>> pid=30894, tid=1082374480 # Error: Possible safepoint reached
>>>>>>>>>> by thread that does not allow it # # JRE version: 7.0 # Java VM:
>>>>>>>>>> OpenJDK 64-Bit Server VM (17.0-b07291505-internal-debug mixed
>>>>>>>>>> mode linux-amd64 ) # If you would like to submit a bug report,
>>>>>>>>>> please visit: # http://java.sun.com/webapps/bugreport/crash.jsp
>>>>>>>>>> # --------------- T H R E A D --------------- Current thread
>>>>>>>>>> (0x00000000012ed000): JavaThread "RMI TCP Connection(idle)"
>>>>>>>>>> daemon [_thread_in_Java, id=30929,
>>>>>>>>>> stack(0x000000004073b000,0x000000004083c000)] Stack:
>>>>>>>>>> [0x000000004073b000,0x000000004083c000], sp=0x0000000040839d60,
>>>>>>>>>> free space=3fb0000000000000000k Native frames: (J=compiled Java
>>>>>>>>>> code, j=interpreted, Vv=VM code, C=native code) V
>>>>>>>>>> [libjvm.so+0x8cad18] V [libjvm.so+0x8cbe38] V
>>>>>>>>>> [libjvm.so+0x4039f1] V [libjvm.so+0x87a183] V
>>>>>>>>>> [libjvm.so+0x8e1127] V [libjvm.so+0x4d52ab] V
>>>>>>>>>> [libjvm.so+0x7ee63d] j sun.misc.Unsafe.park(ZJ)V+0 j
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> java.util.concurrent.locks.LockSupport.parkNanos(Ljava/lang/Object;J)V+20
>>>>>>>>>> j
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Ljava/util/concurrent/SynchronousQueue$TransferStack$SNode;ZJ)Ljava/util/concurrent/SynchronousQueue$TransferStack$SNode;+174
>>>>>>>>>> j
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> java.util.concurrent.SynchronousQueue$TransferStack.transfer(Ljava/lang/Object;ZJ)Ljava/lang/Object;+102
>>>>>>>>>> j
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> java.util.concurrent.SynchronousQueue.poll(JLjava/util/concurrent/TimeUnit;)Ljava/lang/Object;+11
>>>>>>>>>> j
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.getTask()Ljava/lang/Runnable;+141
>>>>>>>>>> j
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+17
>>>>>>>>>> j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5 j
>>>>>>>>>> java.lang.Thread.run()V+11 v ~StubRoutines::call_stub V
>>>>>>>>>> [libjvm.so+0x555ecc] V [libjvm.so+0x747a34] V
>>>>>>>>>> [libjvm.so+0x554b55] V [libjvm.so+0x5551f0] V
>>>>>>>>>> [libjvm.so+0x5553cc] V [libjvm.so+0x5be6cd] V
>>>>>>>>>> [libjvm.so+0x879869] V [libjvm.so+0x87c1b5] V
>>>>>>>>>> [libjvm.so+0x750455] .... (above is the stack, please see the
>>>>>>>>>> attachment for more information) Tony (Xiaohua Guan) On Thu, Jul
>>>>>>>>>> 29, 2010 at 8:07 PM, David Holmes <David.Holmes at oracle.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Tony,
>>>>>>>>>>>
>>>>>>>>>>> I don't see anything obviously wrong with what you attempted.
>>>>>>>>>>> Can you show the full stack dump from the error.
>>>>>>>>>>>
>>>>>>>>>>> David Holmes
>>>>>>>>>>>
>>>>>>>>>>> Tony Guan said the following on 07/30/10 07:29:
>>>>>>>>>>>>
>>>>>>>>>>>> Dear all,
>>>>>>>>>>>>
>>>>>>>>>>>> I want to invoke the GC at a certain time, for example, when
>>>>>>>>>>>> a certain method is called at runtime, so I created a new
>>>>>>>>>>>> subtype of VM_GC_Operation class, and initialized it with an
>>>>>>>>>>>> object op, finally, executed using:VMThread::execute(&op)
>>>>>>>>>>>>
>>>>>>>>>>>> then I got the following error message:
>>>>>>>>>>>>
>>>>>>>>>>>> # Internal Error
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> (/home/tony/software/OpenJDK/jdk7/hotspot/src/share/vm/runtime/thread.cpp:777),
>>>>>>>>>>>> pid=29652, tid=1096460624 # Error: Possible safepoint
>>>>>>>>>>>> reached by thread that does not allow it
>>>>>>>>>>>>
>>>>>>>>>>>> The reason for the fail is that
>>>>>>>>>>>> Thread::check_for_valid_safepoint_state() didn't pass because
>>>>>>>>>>>> the currentThread is not a VMThread.
>>>>>>>>>>>>
>>>>>>>>>>>> My question is: how do I bring the current thread to a
>>>>>>>>>>>> safepoint? or how do I call a GC operation properly? is there
>>>>>>>>>>>> any prerequisite for doing so?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>
>>>>>>>>>>>> Tony (Xiaohua Guan)
>>>>>
>>>>
>>>
>
>
More information about the hotspot-runtime-dev
mailing list