Code review request: CR 6995781 Native Memory Tracking Phase 1
Zhengyu Gu
zhengyu.gu at oracle.com
Thu Jun 28 06:51:05 PDT 2012
Hi Vitaly,
We did, there were concerns over increasing binary sizes.
Thanks,
-Zhengyu
On 6/28/2012 9:44 AM, Vitaly Davidovich wrote:
>
> Hi Zhengyu,
>
> Have you considered using compiler macros (e.g. _func_, _FUNCTION_) to
> record the name of the func that allocated the memory, rather than
> stack walking? Not sure if all compilers that you guys use support
> this, but I think the major ones do.
>
> This would sidestep the inlining issue.
>
> Thanks
>
> Sent from my phone
>
> On Jun 28, 2012 8:18 AM, "Zhengyu Gu" <zhengyu.gu at oracle.com
> <mailto:zhengyu.gu at oracle.com>> wrote:
>
> Hi David,
>
> 2. In jvmg.make on all platforms we unconditionally add
> -D_NMT_NOINLINE_ to inform NMT that no inlining is
> done by the
> compiler. How do we know that? Is there some specific
> compiler flag
> that controls this? If so the setting of
> -D_NMT_NOINLINE_ should be
> conditional on that flag being set. As you can
> override flag settings
> at build-time it would seem possible to me set
> -D_NMT_NOINLINE_
> incorrectly.
>
> It is just a hint to NMT to adjust how many frames it
> needs to walk to
> get proper callsite pc. I can not tell if the flag works
> on all
> platforms with all possible compiler, but seems to work
> for the
> platforms I tested (windows, linux and solaris). I will
> file a RFE for
> build-time override.
>
>
> I'm not sure I made my point clearly on this - as your
> suggested RFE is not what I was suggesting. This would seem
> very compiler-settings specific, yet it is set
> unconditionally, for jvmg builds. Are you assuming that the
> optimization levels set for jvmg builds render this assumption
> valid? What happens if the assumption is wrong? Would we crash
> or just get inaccurate information?
>
> So far, I can tell, it just results inaccurate information. Even
> for optimized build, there is not way to tell if a function
> actually will be inlined or not, so some functions that are
> assumed to be inlined by NMT, are not always inlined. I see the
> callsites not always the same on different platforms, but have yet
> see crashes due to stack walking.
>
> I ran NMT through JPRT and other tests with tracking level =
> detail, which means to perform stack walking for callsites, I have
> not seen any crashes related to stack walking for a fair long time
> (3 ~ 4 months)
>
> But again, that just small set of compilers, which we use ...
>
> Thanks,
>
> -Zhengyu
>
> General comment which I'm sure you've had before: it
> is a real shame
> that every NEW/FREE_C_HEAP_ARRAY etc had to be
> modified to add a tag -
> and it seems the majority of them are mtInternal.
> Could we not define
> the existing macros to add mtInternal, and then define
> new macros, eg
> NEW_C_HEAP_ARRAY_TAGGED, for explicit tagging for use
> when not
> mtInternal?
>
> New macros can be a solution.
>
>
> Thanks for considering. I know there's a lot of schedule
> pressure on this work.
>
> David
> -----
>
> Thanks,
>
> -Zhengyu
>
> Thanks,
> David
> -----
>
> On 28/06/2012 11:32 AM, Zhengyu Gu wrote:
>
> Hi,
>
> Sorry for the long delay, here is the updated webrev:
> http://cr.openjdk.java.net/~zgu/6995781/webrev.01
> <http://cr.openjdk.java.net/%7Ezgu/6995781/webrev.01>
>
> Thanks for many people who take time to review NMT
> code, the new webrev
> reflects the comments and suggestions from you,
> along with the fixes of
> serviceability agent and some bugs.
>
> Thanks,
>
> -Zhengyu
>
>
>
>
>
>
> On 6/12/2012 12:11 PM, Zhengyu Gu wrote:
>
> This is the webrev for native memory tracking
> phase 1, which is
> tracked by CR6995781
> (http://monaco.us.oracle.com/detail.jsf?cr=6995781)
> and related DCmd
> CR7151532
> (http://monaco.us.oracle.com/detail.jsf?cr=7151532).
>
> Native memory tracking (NMT) phase 1, is
> designed to track native
> memory (malloc'd and mmap'd) usages by VM code
> only, it does not track
> the memory usages by libraries and JNI code.
>
> On the implementation side, NMT intercepts
> memory related calls in os
> implementation, such as os::malloc,
> os::realloc, os::free,
> os::reserve_memory, os::commit_memory and etc.
> the caller is required
> to provide the 'memory type' information,
> which represents the VM
> subsystem that the memory is allocated for.
> The 'memory type' is
> defined in /src/share/memory/allocation.hpp.
> Also, a caller's pc is
> captured if NMT tracking level is set to detail.
>
> There are a few ways to tag a memory block to
> a memory type:
>
> 1. os::malloc takes an extra parameter for the
> memory type.
> 2. CHeapObj now is a template, it takes a
> memory type as template
> parameter to tag the object to a specified
> memory type.
> 3. Utility classes, such as Arena,
> GrowableArray and etc. the memory
> type is a parameter of 'new' operator.
> 4. Virtual memory block is tagged on its base
> address.
>
> When NMT intercepts the call, a memory record
> is created and
> serialized by a sequence number, generated by
> global sequence
> generator. The memory record is written to a
> per-thread recorder
> without a lock, when calling thread is a
> 'safepoint visible'
> JavaThread - a JavaThread that will block at
> safepoint. Otherwise,
> ThreadCritical lock is acquired to write to a
> global recorder. The
> recorders (or raw data) are synchronized at
> some safepoints, and
> global sequence generator is also reset at
> synchronization time, to
> avoid overflow the sequence number.
>
> A dedicated NMT worker thread is created to
> process the raw memory
> records. It first stages a generation of raw
> data (a generation is
> defined as span between resets of global
> sequence number), then
> promotes the staged data to a global snapshot,
> which maintains
> information of all 'live' memory block.
>
> NMT Tracking option has to be specified
> through VM command line option
> -XX:NativeMemoryTracking (off by default),
> tracking level can not be
> altered during runtime, but it can be shutdown
> through jcmd tool.
>
> NMT DCmd implements a set of sub-command to
> control NMT runtime and
> request tracking data.
>
> Webrev:
> http://cr.openjdk.java.net/~zgu/6995781/webrev.00
> <http://cr.openjdk.java.net/%7Ezgu/6995781/webrev.00>
>
> The tests and results:
>
> - Passed internal smoke tests
> - JTreg test:
> http://cr.openjdk.java.net/~zgu/6995781/JTreport/
> <http://cr.openjdk.java.net/%7Ezgu/6995781/JTreport/>
>
>
> Additional notes:
> - Some classes' new operators are marked
> _NOINLINE_ (for the same
> purpose as above), the performance runs, I
> performed, indicate that
> they do not impact performance numbers.
>
> - SA agent changes are in progress
>
> Thanks,
>
> -Zhengyu
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20120628/a6f79bd1/attachment.html
More information about the hotspot-dev
mailing list