RFR 8204552: NMT: Separate thread stack tracking from virtual memory tracking

[Thomas Stüfe]

>
> <snip>
> However, I am not even sure if we need all that machinery just to track
> stacks allocation sites. Is that so exciting? I would assume to see always
> the same stacks, since there are not many variations of "start java
> thread". Sure it is cool to see. But to me a summary number "sum of java
> stack memory" would be sufficient, maybe in Threads beside the java thread
> counter. One advantage this would have is that we do not need NMT to be on
> to track that.
>
>
Scratch that part, I did not think this thru. If we forgo keeping track of
individual thread stack allocations in favor of a simple summary we loose
the ability to scan for committed pages and find out how much thread stack
space is actually used. If we want to keep this, and it works accurately,
we need to keep track of thread stacks individually.

..Thomas


>
> On Mon, Feb 25, 2019 at 6:50 PM <zgu at redhat.com> wrote:
>
>> On Mon, 2019-02-25 at 18:21 +0100, Thomas Stüfe wrote:
>> > Short question, just to clarify things:
>> >
>> > when you talk about "virtual memory thread stack" and "malloced
>> > thread stack" you mean actually
>> > a) a thread stack whose boundaries are page aligned and which can
>> > have uncommitted portions and we are able to find that out (e.g.
>> > mincore)
>> > b) a thread stack whose boundaries are not necessarily page aligned
>> > and which either have no uncommitted portions or we are unable to
>> > find that out?
>> >
>> > And the whole "malloc" terminology just stems from the fact that you
>> > use the malloc tracking mechanics of NMT to track the latter, just
>> > because it is a good fit?
>> No, I do mean if the thread stack is backed by virtual memory or
>> malloc'd memory.
>>
>> The thread stack is backed by malloc'd memory, it does not expect to
>> have guard pages. If it is not the case, then this patch is still
>> broken.
>>
>> Thanks,
>>
>> -Zhengyu
>>
>>
>> >
>> > ..Thomas
>> >
>> >
>> > On Mon, Feb 25, 2019 at 5:10 PM <zgu at redhat.com> wrote:
>> > > Friendly ping!
>> > >
>> > > Thanks,
>> > >
>> > > -Zhengyu
>> > >
>> > > On Mon, 2019-02-18 at 14:57 -0500, zgu at redhat.com wrote:
>> > > > On Mon, 2019-02-18 at 08:50 -0500, zgu at redhat.com wrote:
>> > > > > NMT tracks thread stacks as virtual memory since day one, as
>> > > most
>> > > > > of
>> > > > > platofrms are using virtual memory to back thread stacks.
>> > > However,
>> > > > > there are exceptions, e.g. AIX. The matter of fact, POSIX
>> > > standard
>> > > > > does
>> > > > > not dictate what kind of memory or even alignment requirement
>> > > for
>> > > > > thread stacks.
>> > > > >
>> > > > > This patch separates thread stack tracking from virtual memory
>> > > > > tracking. It delegates to virtual memory tracker if thread
>> > > stack is
>> > > > > backed by virtual memory.
>> > > > >
>> > > > > For thread stacks that are backed by malloc'd memory, it can
>> > > not
>> > > > > simply
>> > > > > utilize malloc tracker for detail tracking, since it can not
>> > > > > install
>> > > > > tracking header in place. Therefore, it has to track those
>> > > memory
>> > > > > separately, then piggybacks to existing malloc tracking
>> > > > > infrastructure
>> > > > > during snapshot, for baselining and reporting.
>> > > > >
>> > > > > Thanks for Goetz Lindenmaier and Thomas Stüfe's help to run the
>> > > > > patch
>> > > > > through SAP buidling and testing infrastructure, that is
>> > > especially
>> > > > > helpful for verifying the fix for AIX.
>> > > > >
>> > > > >
>> > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8204552
>> > > > > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8204552/webrev.01/
>> > > > >
>> > > > > Test:
>> > > > >   hotspot_nmt on Linux x64 (fastdebug and release)
>> > > >
>> > > > Also passed JDK-submit tests.
>> > > >
>> > > > Thanks,
>> > > >
>> > > > -Zhengyu
>> > > >
>> > > > >
>> > > > > Thanks,
>> > > > >
>> > > > > -Zhengyu
>> > > > >
>> > > > >
>> > > > >
>>
>