Providing users with thread type
Jean Christophe Beyler
jcbeyler at google.com
Sat Apr 27 03:18:47 UTC 2019
Ok I figured out what was going on. It's not as weird as I thought it would
be: this is seemingly something "new" because it is true for recent JDKs
but not for JDK8 that I was using for my tests.
It still feels a bit wrong and a bit brittle, but if we don't think we
would like to expose something explicitly, then this is already a starting
point I believe, I'll continue testing and see if I can get it working
entirely in my JDK11 agent version.
Thanks all for your input,
Jc
On Sat, Apr 20, 2019 at 11:25 PM Thomas Stüfe <thomas.stuefe at gmail.com>
wrote:
>
>
> On Sun, Apr 21, 2019 at 3:01 AM Jean Christophe Beyler <
> jcbeyler at google.com> wrote:
>
>> Ahh there lies the problem of understanding then :)
>>
>> We can provide that for our users that can use a non-vanilla OpenJDK.
>> Vanilla OpenJDK cannot use that kind of thing since there is no "C" style
>> symbol you could dlsym to. I don't think there is a good mechanism to
>> currently piece the information together using a vanilla OpenJDK.
>>
>> So this conversation is about providing a mechanism for all OpenJDK agent
>> writers to figure this out, even if they had to jump through hoops. The
>> pthread_getname_np would have been a good one presumably but it seems that
>> a lot of systems just rename the native threads for some reason...
>>
>
> I would like to understand this better. From user code, the way to rename
> a thread is to call pthread_setname_np on it, for which you need the
> pthread_t of the thread, and how would user code get that for a VM internal
> thread that never yields control to the outside world? Or do you think this
> renaming is done at kernel/libC level?
>
> Might also be that we have an error somewhere and do not always set the
> thread name. AFAIK we do not have tests for that, so that could bitrot.
>
> ..Thomas
>
>
>>
>> Hence I was offering to either extend AsyncGetCallTrace method (or create
>> a side-one) or extend the JVMPI_CallTrace structure. Either of these would
>> allow profilers to gather this information on future vanilla OpenJDKs. If
>> that makes sense,
>> Jc
>>
>> On Sat, Apr 20, 2019 at 6:14 PM David Holmes <david.holmes at oracle.com>
>> wrote:
>>
>>> Hi Jc,
>>>
>>> On 21/04/2019 8:15 am, Jean Christophe Beyler wrote:
>>> > Hi David,
>>> >
>>> > Hopefully this gives more details :)
>>> >
>>> > Basically, in the Java agent, we set up a SIGPROF handler and we get a
>>> > handler call at a set frequency rate (generally the profiling is for
>>> 30
>>> > seconds). The thread that is doing the signal handler can be a thread
>>> > from the JVM and as we have said in this thread, internal VM threads
>>> > can't get a JNIEnv or are visible to JVMTI anyway so we can't really
>>> get
>>> > the name.
>>>
>>> And yet ...
>>>
>>> > Our current implementation, when faced with a thread like that, calls
>>> a
>>> > method like above that we added in the JVM to provide information
>>> about
>>> > what type of work was being done so that when we provide our profiles,
>>> > we can give that information to the user.
>>>
>>> ... they can call this method you added to the JVM? How do they do that?
>>> And if they can do that why can't they get the name the same way? Or
>>> call any existing query methods to piece together the information?
>>>
>>> Cheers,
>>> David
>>>
>>> > However for agents that cannot use the modified JVM, we cannot provide
>>> > this information it seems and basically we have to fallback to getting
>>> > the PC and just attributing it to libjvm.so (see an example here
>>> >
>>> https://github.com/GoogleCloudPlatform/cloud-profiler-java/blob/master/src/profiler.cc#L127
>>> )
>>> >
>>> > So basically, in cases where we are in a signal handler, where we are
>>> > asynchronous and cannot do a lot, getting a side method to just say :
>>> > "hey is this a compiler thread, is this a GC thread, etc." would
>>> > actually be helpful information during profiling instead of the
>>> general
>>> > "well this many threads were in libjvm.so but we don't know what was
>>> > going on".
>>> >
>>> > I've thought about using some other information such as
>>> > CompilationMXBean to try to assess what is going on during profiling
>>> but
>>> > it seems hard to correctly attribute that back into the actual
>>> profiles
>>> > and untangle the libjvm.so buccket.
>>> >
>>> > Hopefully this makes sense,
>>> > Jc
>>> >
>>> > On Sat, Apr 20, 2019 at 12:20 AM David Holmes <david.holmes at oracle.com
>>> > <mailto:david.holmes at oracle.com>> wrote:
>>> >
>>> > On 20/04/2019 10:29 am, Jean Christophe Beyler wrote:
>>> > > Hi David,
>>> > >
>>> > > On Fri, Apr 19, 2019 at 6:49 PM David Holmes
>>> > <david.holmes at oracle.com <mailto:david.holmes at oracle.com>
>>> > > <mailto:david.holmes at oracle.com
>>> > <mailto:david.holmes at oracle.com>>> wrote:
>>> > >
>>> > > Hi Jc,
>>> > >
>>> > > On 20/04/2019 12:30 am, Jean Christophe Beyler wrote:
>>> > > > Problem is that if I don't have a jthread, I can't get
>>> the
>>> > name it
>>> > > > seems. Perhaps it could help if I gave more information:
>>> > > >
>>> > > > - In our JVM profiling mechanism, we have a SIGPROF (and
>>> maybe
>>> > > that's a
>>> > > > wrong approach :-)) that gets cycled across threads
>>> (some Java
>>> > > threads,
>>> > > > some are the other threads)
>>> > > > - It is the other threads that I'm interested here to
>>> > be able to
>>> > > > distinguish what they are in terms of of profiles
>>> > > >
>>> > > > Is there any way we could provide that (not in JVMTI
>>> then)?
>>> > > > - The only way I could imagine perhaps doing this
>>> would be
>>> > > perhaps
>>> > > > to have a set of other tools at the same time running
>>> > (either using
>>> > > > beans before/after or JFR) but this seems crude as well
>>> > (better than
>>> > > > nothing though)
>>> > > > - I wish there was a way to just be able to get a
>>> type
>>> > for those
>>> > > > internal frames while doing the actual SIGPROF handling
>>> > > >
>>> > > > FWIW, the method we expose basically is like this:
>>> > > > Thread* current_thread =
>>> > > ThreadLocalStorage::get_thread_async_safe();
>>> > >
>>> > > We have Thread::current_or_null_safe() for that.
>>> > >
>>> > >
>>> > > A decade old code might have rotted a bit (or been wrong from
>>> the
>>> > > start), I'll change this internal code :)
>>> > >
>>> > >
>>> > > > if (current_thread == NULL) {
>>> > > > return -1;
>>> > > > } else if (current_thread->is_Compiler_thread()) {
>>> > > > return _activity_compile;
>>> > > > } else if (current_thread->is_Java_thread()) {
>>> > > > return -1;
>>> > > > } else if (current_thread->is_GC_task_thread()) {
>>> > > > return _activity_gc;
>>> > > > } else if (current_thread->is_VM_thread()) {
>>> > > > VMThread* vm_thread = (VMThread*) current_thread;
>>> > > > VM_Operation* vm_op = vm_thread->vm_operation();
>>> > > > if (vm_op != NULL) {
>>> > > > switch (vm_op->type()) {
>>> > > > case VM_Operation::VMOp_GC_HeapInspection:
>>> > > > case VM_Operation::VMOp_GenCollectFull:
>>> > > > case
>>> VM_Operation::VMOp_GenCollectFullConcurrent:
>>> > > > case
>>> VM_Operation::VMOp_GenCollectForAllocation:
>>> > > > case
>>> VM_Operation::VMOp_ParallelGCFailedAllocation:
>>> > > > case VM_Operation::VMOp_ParallelGCSystemGC:
>>> > > > case VM_Operation::VMOp_CGC_Operation:
>>> > > > case VM_Operation::VMOp_CMS_Initial_Mark:
>>> > > > case VM_Operation::VMOp_CMS_Final_Remark:
>>> > > > case VM_Operation::VMOp_G1CollectFull:
>>> > > > case
>>> VM_Operation::VMOp_G1CollectForAllocation:
>>> > > > case VM_Operation::VMOp_G1IncCollectionPause:
>>> > > > return _activity_gc;
>>> > > > default:
>>> > > > break;
>>> > > > }
>>> > > > }
>>> > > > }
>>> > > > return _activity_other_vm;
>>> > > > }
>>> > >
>>> > > So it's not really the thread "type" but the logical
>>> > "activity". For
>>> > > "type" you'd just need a query version of
>>> > Thread::print_on_error (more
>>> > > or less).
>>> > >
>>> > >
>>> > > Not at all sure where you could put this - nor clear why you
>>> > need to
>>> > > put
>>> > > it somewhere: isn't this just something executed by your
>>> SIGPROF
>>> > > handler?
>>> > >
>>> > >
>>> > > Well problem is that I'm not in the JVM at the sigprof handler
>>> > level.
>>> > > I'm actually in the agent,
>>> >
>>> > Not sure what you mean. I'm assuming you're sending a SIGPROF to
>>> each
>>> > thread and using the handler for profiling - no? Otherwise please
>>> > clarify what is happening in each thread.
>>> >
>>> > > so basically from the agent's point of view,
>>> > > I don't really know what "Activity" I just stopped but would
>>> love to
>>> > > know. We added internally this change to figure it out but I'd
>>> > like to
>>> > > get it in the open-source so that all could use it and not just
>>> us
>>> >
>>> > A "char* Thread::get_thread_type()" API might be useful (as I said
>>> a
>>> > query version of print_on_error(). But otherwise this seems
>>> something
>>> > peculiar to your agent so simply composing existing API calls - as
>>> you
>>> > outline - seems the appropriate way to deal with this.
>>> >
>>> > Seems to me the VMThread is the problem here because you want to
>>> try
>>> > and
>>> > attribute the VM_operation to different "buckets". But given the
>>> > bulk of
>>> > the work is actually done by other threads (e.g. GC), and going
>>> forward
>>> > less and less will be done by the VMThread itself (e.g. async
>>> monitor
>>> > deflation), is it really worth trying to classify this at a finer
>>> level
>>> > than just "in the VM"?
>>> >
>>> > Cheers,
>>> > David
>>> >
>>> > > internally. Basically, like I said ,when using open-source
>>> > profilers,
>>> > > this would help divide up the "libjvm.so" bucket that a lot of
>>> > profilers
>>> > > are getting.
>>> > > Thanks for your insight as always,
>>> > > Jc
>>> > >
>>> > >
>>> > >
>>> > > David
>>> > >
>>> > > > It's crude but we believe it is effective to at least
>>> > "bucketize"
>>> > > the
>>> > > > internals while doing our profiling.
>>> > > >
>>> > > > Thanks for your input,
>>> > > > Jc
>>> > > >
>>> > > > On Fri, Apr 19, 2019 at 9:01 AM Alan Bateman
>>> > > <Alan.Bateman at oracle.com <mailto:Alan.Bateman at oracle.com>
>>> > <mailto:Alan.Bateman at oracle.com <mailto:Alan.Bateman at oracle.com>>
>>> > > > <mailto:Alan.Bateman at oracle.com
>>> > <mailto:Alan.Bateman at oracle.com>
>>> > > <mailto:Alan.Bateman at oracle.com
>>> > <mailto:Alan.Bateman at oracle.com>>>> wrote:
>>> > > >
>>> > > > On 19/04/2019 00:12, David Holmes wrote:
>>> > > > >
>>> > > > > I think it would be difficult to put something
>>> like
>>> > this
>>> > > in JVM TI
>>> > > > > given that the use of threads within the JVM are
>>> > purely an
>>> > > > > implementation detail and not standardized in any
>>> > way. And
>>> > > many of
>>> > > > > those threads are hidden from JVM TI anyway.
>>> > > > >
>>> > > > > The names of threads are the normal way to see
>>> what
>>> > "type"
>>> > > of thread
>>> > > > > you're dealing with.
>>> > > > Right, JVM TI only deals with "Java threads" (jthread
>>> > object) and
>>> > > > has no
>>> > > > knowledge about other threads. It might be possible
>>> to
>>> > use its
>>> > > > extension
>>> > > > mechanism to provide information about other threads
>>> > but it
>>> > > wouldn't be
>>> > > > interoperable with anything that use jtherad objects.
>>> > > >
>>> > > > -Alan
>>> > > >
>>> > > >
>>> > > >
>>> > > > --
>>> > > >
>>> > > > Thanks,
>>> > > > Jc
>>> > >
>>> > >
>>> > >
>>> > > --
>>> > >
>>> > > Thanks,
>>> > > Jc
>>> >
>>> >
>>> >
>>> > --
>>> >
>>> > Thanks,
>>> > Jc
>>>
>>
>>
>> --
>>
>> Thanks,
>> Jc
>>
>
--
Thanks,
Jc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20190426/61a73ecd/attachment-0001.html>
More information about the serviceability-dev
mailing list