Garbage Collection tuning advice ?

[Rohit Mohta]

Hi,

  We are facing similar issue for our high performance application. We
noticed, JVM ergonomic took over and adjusted the values to meet its goal
(adaptive policy). This worked in most scenario, but caused problems in
some case.

My recommendations will be

a) Turn on GC logging (if not yet done) either for default setting or the
one you have mentioned

b) Check if you have a regular premature movement to old generation. If
yes, you might want to set a specific new size and/or survivor size.

c) You have 16GB ram but have set Max heap as 8GB. If you are getting OOM,
increase Xmx.
On 20 Apr 2015 18:50, "Nicola Abello" <nicola at infielddesign.com> wrote:

> Hello everyone, I am currently using the Adobe Experience Manager for a
> Client's site (Java language). It uses openJDK:
>
> #*java -version*
>
> *java version "1.7.0_65"*
> *OpenJDK Runtime Environment (rhel-2.5.1.2.el6_5-x86_64 u65-b17)*
> *OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode)*
>
> It is running on Rackspace with the following:
>
> *vCPU:* *4*
> *Memory:* *16GB*
> *Guest OS:* *Red Hat Enterprise Linux 6 (64-bit)*
>
> Since it has been in production I have been experiencing very slow
> performance on the part of the application. It goes like this I launch the
> app, everything is smooth then 3 to 4 days later the CPU usage spikes to
> 400% (~4000 users/day hit the site). I got a few OOM exceptions (1 or 2)
> but mostly the site was exceptionally slow and never becomes an OOM
> exception. Since I am a novice at Java Memory management I started reading
> about how it works and found tools like jstat. When the system was
> overwhelmed the second time around, I ran:
>
> #*top*
>
> Got the PID of the java process and then pressed shift+H and noted the
> PIDs of the threads with high CPU percentage. Then I ran
>
> #*sudo -uaem jstat <PID>*
>
> Got a thread dump and converted the thread PIDs I wrote down previously
> and searched for their hex value in the dump. After all that, I finally
> found that it was not surprisingly the Garbage Collector that is flipping
> out for some reason.
>
> I started reading a lot about Java GC tuning and came up with the
> following java options.
> So restarted the application with the following options:
>
> *java* -Dcom.day.crx.persistence.tar.IndexMergeDelay=0
>  -Djackrabbit.maxQueuedEvents=1000000
>  -Djava.io.tmpdir=/srv/aem/tmp/
>
> * -XX:+HeapDumpOnOutOfMemoryError *
> * -XX:HeapDumpPath=/srv/aem/tmp/ *
> * -Xms8192m -Xmx8192m *
> * -XX:PermSize=256m *
> * -XX:MaxPermSize=1024m *
> * -XX:+UseParallelGC *
> * -XX:+UseParallelOldGC *
> * -XX:ParallelGCThreads=4 *
> * -XX:NewRatio=1*
>
>  -Djava.awt.headless=true
>  -server
>  -Dsling.run.modes=publish
>  -jar crx-quickstart/app/cq-quickstart-6.0.0-standalone.jar start
> -c crx-quickstart -i launchpad -p 4503
>  -Dsling.properties=conf/sling.properties
>
> And it looks like it is performing much better but I think that it
> probably needs more GC tuning.
>
> When I run:
>
> #*sudo -uaem jstat <PID> -gcutils*
>
> I get this:
>
> S0     S1     E       O       P      YGC      YGCT     FGC     FGCT      GCT
>
> 0.00   0.00  55.97 100.00  45.09    4725   521.233   505  4179.584
> 4700.817
>
> after 4 days that I restarted it.
>
> When I run:
>
> #*sudo -uaem jstat <PID> -gccapacity*
>
> I get this:
>
>  NGCMN     NGCMX      NGC      S0C   S1C        EC
> 4194304.0  4194304.0  4194304.0 272896.0 279040.0 3636224.0
>
>  OGCMN   OGCMX  OGC                OC       PGCMN           PGCMX
> 4194304.0  4194304.0   4194304.0   4194304.0 262144.0  1048576.0
>
> PGC        PC     YGC     FGC
> 262144.0 262144.0   4725    509
>
> after 4 days that I restarted it.
>
> These result are much better than when I started but I think it can get
> even better. I'm not really sure what to do next as I'm no GC pro so I was
> wondering if you guys would have any tips or advice for me on how I could
> get better app/GC performance and if anything is obvious like ratio's and
> sizes of youngGen and oldGen ?
>
> How should I set the survivors and eden sizes/ratios ?
> Should I change GC type like use CMS GC or G1 ?
> How should I proceed ?
>
> Any advice would be helpful.
>
> Best,
> Nicola
>
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20150420/8dd0903d/attachment.html>