how to bring jvm to safepoint
Tony Guan
guanxiaohua at gmail.com
Sun Aug 1 13:50:57 PDT 2010
Hi David,
Thanks for the explanation, now I get to understand it.
Have to say, it's a nice failure for me. :)
Tony (Xiaohua Guan)
On Sat, Jul 31, 2010 at 10:43 PM, David Holmes <David.Holmes at oracle.com> wrote:
> Hi Tony,
>
> Now it is all clear :)
>
>> It's exactly how the error occurred, I am monitoring the
>> sun.misc.Unsafe.park() method call with dtrace(no dtrace agent, just
>> intercepting inside jvm implementation), once this method gets called,
>> I will call GenCollectedHeap::invoke_transEden_collection() to start a
>> GC using VMThread::Execute(&op).
>
> Right, you are using:
>
> _ZN13SharedRuntime18dtrace_method_exitEP10JavaThreadP13methodOopDesc+0x255
>
> but dtrace_method_exit is defined as a runtime "leaf" method (JRT_LEAF).
> "leaf" methods are not allowed to perform certain actions and in debug mode
> we have JRT_Leaf_Verifier that is used to check those actions. A
> JRT_Leaf_Verifier is a No_Safepoint_Verifier - hence as I said before you
> get the assertion failure due to this.
>
> When you do the VMThread::execute to initiate the safepoint operation for
> your GC action, the current thread will block waiting for it to complete.
> Part of that blocking involves a safepoint check and hence the system finds
> that the thread is attempting to enter a safepoint from a "leaf" method -
> and that is not allowed.
>
> You might just try changing the definition of dtrace_method_exit to be
> JRT_ENTRY rather than JRT_LEAF, but I don't know if that will have
> unexpected side-effects.
>
> There are lots of rules (not necessarily clearly documented) that control
> how you can call into the VM runtime and how different parts of the VM
> runtime can call other parts of the runtime.
>
> Cheers,
> David Holmes
>
> Tony Guan said the following on 08/01/10 08:10:
>>
>> Hi David & Tom,
>>
>> Please see the decoded stack below(also full decoded log is attached):
>>
>> "
>> Current thread (0x00000000012ed000): JavaThread "RMI TCP
>> Connection(idle)" daemon [_thread_in_Java, id=30929,
>> stack(0x000000004073b000,0x000000004083c000)]
>>
>> Stack: [0x000000004073b000,0x000000004083c000],
>> sp=0x0000000040839d60, free space=3fb0000000000000000k
>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
>> code)
>> V [libjvm.so+0x8cad18];; _ZN7VMError6reportEP12outputStream+0xb72
>> V [libjvm.so+0x8cbe38];; _ZN7VMError14report_and_dieEv+0x5f6
>> V [libjvm.so+0x4039f1];; _Z12report_fatalPKciS0_+0x6b
>> V [libjvm.so+0x87a183];;
>> _ZN6Thread31check_for_valid_safepoint_stateEb+0x33
>> V [libjvm.so+0x8e1127];; _ZN8VMThread7executeEP12VM_Operation+0x49
>> V [libjvm.so+0x4d52ab];;
>> _ZN16GenCollectedHeap27invoke_transEden_collectionEi+0x33
>> V [libjvm.so+0x7ee63d];;
>> _ZN13SharedRuntime18dtrace_method_exitEP10JavaThreadP13methodOopDesc+0x255
>> j sun.misc.Unsafe.park(ZJ)V+0
>> j
>> java.util.concurrent.locks.LockSupport.parkNanos(Ljava/lang/Object;J)V+20
>> j
>> java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Ljava/util/concurrent/SynchronousQueue$TransferStack$SNode;ZJ)Ljava/util/concurrent/SynchronousQueue$TransferStack$SNode;+174
>> j
>> java.util.concurrent.SynchronousQueue$TransferStack.transfer(Ljava/lang/Object;ZJ)Ljava/lang/Object;+102
>> j
>> java.util.concurrent.SynchronousQueue.poll(JLjava/util/concurrent/TimeUnit;)Ljava/lang/Object;+11
>> j
>> java.util.concurrent.ThreadPoolExecutor.getTask()Ljava/lang/Runnable;+141
>> j
>> java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+17
>> j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
>> j java.lang.Thread.run()V+11
>> v ~StubRoutines::call_stub
>> V [libjvm.so+0x555ecc];;
>>
>> _ZN9JavaCalls11call_helperEP9JavaValueP12methodHandleP17JavaCallArgumentsP6Thread+0x474
>> V [libjvm.so+0x747a34];;
>>
>> _ZN2os20os_exception_wrapperEPFvP9JavaValueP12methodHandleP17JavaCallArgumentsP6ThreadES1_S3_S5_S7_+0x32
>> V [libjvm.so+0x554b55];;
>>
>> _ZN9JavaCalls4callEP9JavaValue12methodHandleP17JavaCallArgumentsP6Thread+0x6f
>> V [libjvm.so+0x5551f0];;
>>
>> _ZN9JavaCalls12call_virtualEP9JavaValue11KlassHandle12symbolHandleS3_P17JavaCallArgumentsP6Thread+0x158
>> V [libjvm.so+0x5553cc];;
>>
>> _ZN9JavaCalls12call_virtualEP9JavaValue6Handle11KlassHandle12symbolHandleS4_P6Thread+0x86
>> V [libjvm.so+0x5be6cd];; _Z12thread_entryP10JavaThreadP6Thread+0x9f
>> V [libjvm.so+0x879869];; _ZN10JavaThread17thread_main_innerEv+0xc9
>> V [libjvm.so+0x87c1b5];; _ZN10JavaThread3runEv+0x2c1
>> V [libjvm.so+0x750455];; _Z10java_startP6Thread+0x16f
>> "
>>
>> It's exactly how the error occurred, I am monitoring the
>> sun.misc.Unsafe.park() method call with dtrace(no dtrace agent, just
>> intercepting inside jvm implementation), once this method gets called,
>> I will call GenCollectedHeap::invoke_transEden_collection() to start a
>> GC using VMThread::Execute(&op).
>>
>> At the beginning of VMTHread::Execute(),
>> check_for_valid_safepoint_state is called like this:
>>
>> void VMThread::execute(VM_Operation* op) {
>> Thread* t = Thread::current();
>> if (!t->is_VM_thread()) {
>> // JavaThread or WatcherThread
>> t->check_for_valid_safepoint_state(true); //And we failed here!!
>> // New request from Java thread, evaluate prologue
>> if (!op->doit_prologue()) {
>> return; // op was cancelled
>> }
>> ...
>> }
>>
>> That's why I said the current thread is not a VM_thread. I think I can
>> make a workaround by invoking the GC at the next allocation request,
>> which should be in the safepoint. But would be great if I can learn
>> more about hotspot solving this problem.
>>
>> Thanks a lot!
>>
>> Tony (Xiaohua Guan)
>>
>>
>>
>> On Sat, Jul 31, 2010 at 12:56 AM, David Holmes <David.Holmes at oracle.com>
>> wrote:
>>>
>>> Tom Rodriguez said the following on 07/31/10 05:01:
>>>>
>>>> Actually I think the problem is that the thread is in thread_in_Java
>>>> state. I don't believe you can safepoint from that state, only
>>>> thread_in_vm.
>>>
>>> The reported failure:
>>>
>>> # Error: Possible safepoint reached by thread that does not allow it
>>>
>>> is here:
>>>
>>> void Thread::check_for_valid_safepoint_state(bool potential_vm_operation)
>>> {
>>> // Check if current thread is allowed to block at a safepoint
>>> if (!(_allow_safepoint_count == 0))
>>> fatal("Possible safepoint reached by thread that does not allow it");
>>>
>>> as far as I can see the only code that touches _allow_safepoint_count are
>>> the No_Safepoint_Verifier classes.
>>>
>>> Also the stack shows:
>>>
>>> V [libjvm.so+0x7ee63d]
>>> j sun.misc.Unsafe.park(ZJ)V+0
>>>
>>> and so we should have transitioned to _thread_in_vm here.
>>>
>>> That said, Tony also stated:
>>>
>>>> The reason for the fail is that
>>>> Thread::check_for_valid_safepoint_state() didn't pass because
>>>> the currentThread is not a VMThread.
>>>
>>> But that is not supported by the error produced.
>>>
>>> Tony: why did you make the above comment?
>>>
>>> Either way we need to see a decoded stack to get an idea of what is going
>>> wrong.
>>>
>>> Cheers,
>>> David
>>>
>>>> tom
>>>>
>>>> On Jul 30, 2010, at 12:04 AM, David Holmes <David.Holmes at oracle.com>
>>>> wrote:
>>>>
>>>>> Sorry Tony I keep forgetting that I don't have a way to decode a
>>>>> crash log from a VM that you built yourself. :( However you may be
>>>>> able to decode it yourself please see:
>>>>>
>>>>> http://blogs.sun.com/dave/entry/a_tool_to_decode_hs
>>>>>
>>>>> for a perl script.
>>>>>
>>>>> That said looking at the actual assertion failure I am guessing
>>>>> that you initiated the VM operation from code where a
>>>>> No_Safepoint_verifier is active.
>>>>>
>>>>> David
>>>>>
>>>>> Tony Guan said the following on 07/30/10 15:58:
>>>>>>
>>>>>> Hi David, Thanks a lot! I am copying the content to below, and
>>>>>> for convenience, the full log file is attached. cat
>>>>>> hs_err_pid30894.log # # A fatal error has been detected by the
>>>>>> Java Runtime Environment: # # Internal Error
>>>>>>
>>>>>> (/home/tony/software/OpenJDK/jdk7/hotspot/src/share/vm/runtime/thread.cpp:777),
>>>>>> pid=30894, tid=1082374480 # Error: Possible safepoint reached
>>>>>> by thread that does not allow it # # JRE version: 7.0 # Java VM:
>>>>>> OpenJDK 64-Bit Server VM (17.0-b07291505-internal-debug mixed
>>>>>> mode linux-amd64 ) # If you would like to submit a bug report,
>>>>>> please visit: # http://java.sun.com/webapps/bugreport/crash.jsp
>>>>>> # --------------- T H R E A D --------------- Current thread
>>>>>> (0x00000000012ed000): JavaThread "RMI TCP Connection(idle)"
>>>>>> daemon [_thread_in_Java, id=30929,
>>>>>> stack(0x000000004073b000,0x000000004083c000)] Stack:
>>>>>> [0x000000004073b000,0x000000004083c000], sp=0x0000000040839d60,
>>>>>> free space=3fb0000000000000000k Native frames: (J=compiled Java
>>>>>> code, j=interpreted, Vv=VM code, C=native code) V
>>>>>> [libjvm.so+0x8cad18] V [libjvm.so+0x8cbe38] V
>>>>>> [libjvm.so+0x4039f1] V [libjvm.so+0x87a183] V
>>>>>> [libjvm.so+0x8e1127] V [libjvm.so+0x4d52ab] V
>>>>>> [libjvm.so+0x7ee63d] j sun.misc.Unsafe.park(ZJ)V+0 j
>>>>>>
>>>>>>
>>>>>> java.util.concurrent.locks.LockSupport.parkNanos(Ljava/lang/Object;J)V+20
>>>>>> j
>>>>>>
>>>>>>
>>>>>> java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Ljava/util/concurrent/SynchronousQueue$TransferStack$SNode;ZJ)Ljava/util/concurrent/SynchronousQueue$TransferStack$SNode;+174
>>>>>> j
>>>>>>
>>>>>>
>>>>>> java.util.concurrent.SynchronousQueue$TransferStack.transfer(Ljava/lang/Object;ZJ)Ljava/lang/Object;+102
>>>>>> j
>>>>>>
>>>>>>
>>>>>> java.util.concurrent.SynchronousQueue.poll(JLjava/util/concurrent/TimeUnit;)Ljava/lang/Object;+11
>>>>>> j
>>>>>>
>>>>>>
>>>>>> java.util.concurrent.ThreadPoolExecutor.getTask()Ljava/lang/Runnable;+141
>>>>>> j
>>>>>>
>>>>>>
>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+17
>>>>>> j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5 j
>>>>>> java.lang.Thread.run()V+11 v ~StubRoutines::call_stub V
>>>>>> [libjvm.so+0x555ecc] V [libjvm.so+0x747a34] V
>>>>>> [libjvm.so+0x554b55] V [libjvm.so+0x5551f0] V
>>>>>> [libjvm.so+0x5553cc] V [libjvm.so+0x5be6cd] V
>>>>>> [libjvm.so+0x879869] V [libjvm.so+0x87c1b5] V
>>>>>> [libjvm.so+0x750455] .... (above is the stack, please see the
>>>>>> attachment for more information) Tony (Xiaohua Guan) On Thu, Jul
>>>>>> 29, 2010 at 8:07 PM, David Holmes <David.Holmes at oracle.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Tony,
>>>>>>>
>>>>>>> I don't see anything obviously wrong with what you attempted.
>>>>>>> Can you show the full stack dump from the error.
>>>>>>>
>>>>>>> David Holmes
>>>>>>>
>>>>>>> Tony Guan said the following on 07/30/10 07:29:
>>>>>>>>
>>>>>>>> Dear all,
>>>>>>>>
>>>>>>>> I want to invoke the GC at a certain time, for example, when
>>>>>>>> a certain method is called at runtime, so I created a new
>>>>>>>> subtype of VM_GC_Operation class, and initialized it with an
>>>>>>>> object op, finally, executed using:VMThread::execute(&op)
>>>>>>>>
>>>>>>>> then I got the following error message:
>>>>>>>>
>>>>>>>> # Internal Error
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> (/home/tony/software/OpenJDK/jdk7/hotspot/src/share/vm/runtime/thread.cpp:777),
>>>>>>>> pid=29652, tid=1096460624 # Error: Possible safepoint
>>>>>>>> reached by thread that does not allow it
>>>>>>>>
>>>>>>>> The reason for the fail is that
>>>>>>>> Thread::check_for_valid_safepoint_state() didn't pass because
>>>>>>>> the currentThread is not a VMThread.
>>>>>>>>
>>>>>>>> My question is: how do I bring the current thread to a
>>>>>>>> safepoint? or how do I call a GC operation properly? is there
>>>>>>>> any prerequisite for doing so?
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>> Tony (Xiaohua Guan)
>
More information about the hotspot-runtime-dev
mailing list