Trace object allocation flag proposal
Dmitry Samersoff
Dmitry.Samersoff at oracle.com
Wed Dec 15 11:03:08 PST 2010
Xiaobin,
We have three independent diagnostic systems in hotspot:
1. JVMTI
2. DTrace
3. Conventional logging
It's "not good" from long term perspective and I think we should unify it.
My personal opinion - we should have only JVMTI and make it as
lightweight as possible.
-Dmitry
On 2010-12-15 21:24, Xiaobin Lu wrote:
> One of the problems JVMTI has is that once it is enabled, it will fire
> all kinds of events which will cause even more overheads. Correct me if
> I am wrong here.
>
> DTrace provides obj-alloc probes which is exactly what many people like
> to have on Linux. And that is exactly what I am proposing here.
>
> Thanks,
>
> -Xiaobin
>
> On Wed, Dec 15, 2010 at 10:06 AM, Y. S. Ramakrishna
> <y.s.ramakrishna at oracle.com <mailto:y.s.ramakrishna at oracle.com>> wrote:
>
> Doesn't JVMTI already provide hooks (BCI?) for tracing allocation?
> Perhaps that's found to be too obtrusive? Don't Java IDE's
> use JVMTI to already deliver this kind of info to developers?
>
> Pardon my naivette re JVMTI capabilities in this regard.
> -- ramki
> (java-serviceability-tools-illiterate)
>
>
> On 12/15/10 10:00, Xiaobin Lu wrote:
>
> Thanks for your feedback, David.
>
> Let me try to clarify with some more background information.
>
> One of the problems many application has is the object
> allocation problem (Duh ...). Over releases, GC becomes more and
> more frequent. It is pretty hard for one single team to find out
> where the problem goes wrong. The existing tool such as jhat or
> jmap is very hard to work with giant heap dump file. So the flag
> I propose is mostly for diagnostic purpose, it won't be enabled
> by default in production. Having said so, if we make it as a
> manageable flag, we can turn it on when GC goes wild.
>
> Another value this flag could provide is to find out why
> sometime OOM occurs. For example, someone who wrote a search
> application on top of Lucene framework suddenly found a crash
> due to OOM. The stack trace points to Lucene code which is hard
> to instrument, so this flag could provide insight to why the
> allocation fails. Maybe it is due to a corrupted length value
> passed to "new byte[length];" etc.
>
> I like your idea on per-thread basis, however, for a lot of web
> application, thread comes and go. It is pretty hard to pin point
> on which thread I want to trace the object allocation.
>
> To answer your last question, what I am really interested in
> finding out is what type of object allocation makes GC happens
> more frequent. Randomly taking snapshot of heap using jmap is
> absolutely not an idea way to do so since not only it generates
> a giant file which is difficult to work with, also it will pause
> the application and cause negative impact to the application
> throughput. After I get the type of hot allocation, if it
> happens to be an application level object type such as
> com.Xyz.search.IndexReader, I can instrument the constructor to
> dump the caller stack. People here also suggests it would be
> nice if we could dump the allocation stack trace for some
> particular hot types.
>
> I could propose the diff for your folks to review.
>
> Thanks,
>
> -Xiaobin
>
> On Tue, Dec 14, 2010 at 11:58 PM, David Holmes
> <David.Holmes at oracle.com <mailto:David.Holmes at oracle.com>
> <mailto:David.Holmes at oracle.com
> <mailto:David.Holmes at oracle.com>>> wrote:
>
> Hi Xiaobin,
>
> The problem with tracing like this is that to be useful the
> tracing
> must be unobtrusive and be able to handle getting called
> millions of
> times (which is what will happen in those GC scenarios you
> describe). The sheer volume of data generated as you suggest
> below
> would overwhelm the app and be pretty hard to work with.
>
> Per-thread statistics of particular types (or of objects
> larger than
> a certain size) might be more useful, with a dump able to be
> requested on-demand.
>
> But I think you'd need to be able to combine this with heap dump
> info to be useful.
>
> It really depends on exactly what info you want to be able to
> deduce: simple number of objects of given types, hot allocation
> sites, hot threads, ...
>
> Cheers,
> David
>
> Xiaobin Lu said the following on 12/15/10 17:07:
>
> Hi folks,
>
> I would like to propose a way to trace object allocation on
> Linux. On Solaris, we have DTrace which is pretty nice.
> But on
> Linux, it is almost impossible to do so. Correct me if I am
> wrong here.
>
> So I am thinking to add a manageable VM flag and let's
> call it
> TraceObjectAllocation. When enabled, we can output
> something like:
>
> thread id: 10 class name: java/lang/reflect/Method
> size: 80 bytes
> thread id: 10 class name: [Ljava/lang/Class;
> size: 16 bytes
> thread id: 10 class name: [C
> size: 56 bytes
> thread id: 10 class name: java/lang/reflect/Method
> size: 80 bytes
> thread id: 10 class name: [Ljava/lang/Class;
> size: 16 bytes
>
> As you could imagine, this could be very useful to keep
> track of
> object allocation behavior in the app. Some smart tool
> can take
> advantage of this to print a histogram (like top 10 hot
> allocations) of object allocation. I would like to know your
> thoughts and suggestions on this.
>
> I have a detailed proposal on this attached in PDF file.
>
> Thanks,
>
> -Xiaobin
>
>
>
>
--
Dmitry Samersoff
J2SE Sustaining team, SPB04
* Give Rabbit time and he'll always get the answer ...
More information about the hotspot-runtime-dev
mailing list