java process memory usage is higher than Xmx

[yao]

Hi Thomas,

Thanks for your good explanation, very informational and helpful.

Thanks
-Shengzhe



On Mon, Jan 6, 2014 at 3:19 PM, Thomas Schatzl <thomas.schatzl at oracle.com>wrote:

> Hi Shengzhe,
>
> On Mon, 2014-01-06 at 13:49 -0800, yao wrote:
> > Hi Thomas,
> >         Possibly remembered set size.
> >         Can you enable -XX:+UnlockDiagnosticVMOptions and -XX:
> >         +G1SummarizeRSetStats?
> >         The line
> >           Total heap region rem set sizes = 5256086K.  Max = 8640K.
> >         gives you a good idea about remembered set size memory usage.
> >
> > You are right, rem set occupies ~7GB
>
> Could you add -XX:+G1SummarizeConcMark? The GC then shows some details
> about the work done during cleanup phases at VM exit.
>
> At 7 GB remembered set size most likely the phase that tries to minimize
> the remembered set is dominant.
>
> It should show up as large "RS scrub total time" compared to the "Final
> counting total time".
> >
> >  Concurrent RS processed -1863202596 cards
> >   Of 12780432 completed buffers:
> >      12611979 ( 98.7%) by conc RS threads.
> >        168453 (  1.3%) by mutator threads.
> >   Conc RS threads times(s)
> >           0.00     0.00     0.00     0.00     0.00     0.00     0.00
> > 0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00
> > 0.00     0.00     0.00    \
> >  0.00
> >   Total heap region rem set sizes = 7520648K.  Max = 13256K.
> >   Static structures = 347K, free_lists = 28814K.
> >     141012349 occupied cards represented.
> >     Max size region =
> > 93:(O)[0x00007fcb48000000,0x00007fcb4a000000,0x00007fcb4a000000], size
> > = 13257K, occupied = 2639K.
> >     Did 0 coarsenings.
> >
> >         I copied above line from one of your responses to the "G1 GC
> >         clean up
> >         time is too long" thread, and it seems the remembered set
> >         takes ~5GB
> >         there.
> >
> > I did set -XX:G1RSetRegionEntries=4096 to avoid coarsenings; however,
> > the cleanup time seems not being reduced, it is still around 700
> > milliseconds, althrough there is no coarsenings.
> >
> > Any hint for tuning ? Because I want process with G1 use the same heap
> > as CMS to compare the performance. But I cannot do so if rem set is
>
> You could still compare performance with the same total memory usage.
>
> >  that large, the process will be likely killed by OOM killed if I gave
> > more memory.
>
> The default value of G1RSetRegionEntries at 32M region size should be
> 1536 (= (log(region size) - log(1MB) + 1) * G1RSetRegionEntriesBase by
> default); the chosen value means that you allow G1 to keep a larger
> remembered set (ie. less coarsening) per region.
>
> The "RS scrub" part of cleanup is roughly dependent on remembered set
> size, and this is the main knob to turn here imo.
>
> So it seems that increasing the G1RSetRegionEntries is
> counter-productive for decreasing gc cleanup time, because scrubbing
> coarsened remembered sets looks fast. I do not have numbers though, just
> a feeling.
> Coarsening mostly increases gc pause time (RS Scan time to be exact).
>
> Otoh you mentioned that gc cleanup time did not change when changing
> G1RSetRegionEntries.
>
> It's best to measure where the time is spent using the
> G1SummarizeConcMark switch and then possibly change the
> G1RSetRegionEntries value (and measuring the impact).
>
> Thomas
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20140106/5f6e9078/attachment.html