Continuous Full GC - Java 8

[Rohit Mohta]

Hi All,  Attached GC log file for one our JVM process. Any idea why this is happening? It stopped after we restarted this process.
Following is our observation

*       Any object allocation instruction is triggering a Full GC

*       As all object allocation (memory allocation) is done in Eden Space - for some reason fails continously.

*       We could see 452 Full GC in 2 mins.

*       In the attached log, we could see the below lines. What we notice is, YoungGen has 0K memory allocation, and alternatively we have 64K or so memory allocation.

2017-10-28T20:30:32.207+0000: 382100.678: [Full GC (Allocation Failure) [PSYoungGen: 0K->0K(253952K)] [ParOldGen: 146546K->146546K(348160K)] 146546K->146546K(602112K), [Metaspace: 45647K->45647K(1091584K)], 0.1865769 secs] [Times: user=0.87 sys=0.00, real=0.19 secs]
2017-10-28T20:30:32.401+0000: 382100.872: [Full GC (Allocation Failure) [PSYoungGen: 160K->0K(253952K)] [ParOldGen: 146546K->146547K(348160K)] 146706K->146547K(602112K), [Metaspace: 45647K->45647K(1091584K)], 0.1871330 secs] [Times: user=0.88 sys=0.00, real=0.18 secs]
2017-10-28T20:30:32.592+0000: 382101.064: [Full GC (Allocation Failure) [PSYoungGen: 0K->0K(253952K)] [ParOldGen: 146547K->146547K(348160K)] 146547K->146547K(602112K), [Metaspace: 45647K->45647K(1091584K)], 0.1852237 secs] [Times: user=0.88 sys=0.01, real=0.18 secs]
2017-10-28T20:30:32.786+0000: 382101.257: [Full GC (Allocation Failure) [PSYoungGen: 192K->0K(253952K)] [ParOldGen: 146547K->146558K(348160K)] 146739K->146558K(602112K), [Metaspace: 45647K->45647K(1091584K)], 0.1879624 secs] [Times: user=0.89 sys=0.00, real=0.19 secs]
2017-10-28T20:30:32.979+0000: 382101.450: [Full GC (Allocation Failure) [PSYoungGen: 0K->0K(253952K)] [ParOldGen: 146558K->146552K(348160K)] 146558K->146552K(602112K), [Metaspace: 45647K->45647K(1091584K)], 0.2331467 secs] [Times: user=1.15 sys=0.00, real=0.24 secs]

Exception in std_out:
java.lang.OutOfMemoryError: Java heap space
        at java.nio.HeapByteBuffer.<init>(Unknown Source) ~[?:1.8.0_121]
        at java.nio.ByteBuffer.allocate(Unknown Source) ~[?:1.8.0_121]

Similar stack trace is also observed for a plain object allocation instructions.


System Details:

*      Java HotSpot(TM) 64-Bit Server VM (25.121-b13) for linux-amd64 JRE (1.8.0_121-b13), built on Dec 12 2016 16:36:53 by "java_re" with gcc 4.3.0 20080428 (Red Hat 4.3.0-8)

*      OS: Red Hat Enterprise Linux Server release 7.3 (Maipo)




Thank You
Rohit
GAN Development
Office         : +1-602-537-8983
Mail To       : rohit.mohta at aexp.com<mailto:rohit.mohta at aexp.com> (Or ping in Slack)
Location     : American Express Desert Ridge, OB2 - 3rd Floor Near C6, 18850 N 56th Street Phoenix, AZ 85054

"if you can't explain it simply you don't understand it well enough" - Richard Feynman (maybe?)


*** AXP - INTERNAL ***



American Express made the following annotations
******************************************************************************
"This message and any attachments are solely for the intended recipient and may contain confidential or privileged information. If you are not the intended recipient, any disclosure, copying, use, or distribution of the information included in this message and any attachments is prohibited. If you have received this communication in error, please notify us by reply e-mail and immediately and permanently delete this message and any attachments. Thank you."

American Express a ajouté le commentaire suivant le Ce courrier et toute pièce jointe qu'il contient sont réservés au seul destinataire indiqué et peuvent renfermer des 
renseignements confidentiels et privilégiés. Si vous n'êtes pas le destinataire prévu, toute divulgation, duplication, utilisation ou distribution du courrier ou de toute pièce jointe est interdite. Si vous avez reçu cette communication par erreur, veuillez nous en aviser par courrier et détruire immédiatement le courrier et les pièces jointes. Merci.

******************************************************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20171030/3272f1ea/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gc-28Oct2017.log.log.zip
Type: application/x-zip-compressed
Size: 78285 bytes
Desc: gc-28Oct2017.log.log.zip
URL: <http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20171030/3272f1ea/gc-28Oct2017.log.log-0001.zip>