Strange isssue about OOM

Wed Jan 16 06:14:05 PST 2008

Hi Ramki,

Thanks for your explanation about "HandlePromotionFailure" option.

>From the gc log,
25210.819: [GC 25210.820: [DefNew: 86804K->1788K(169600K), 0.0486894
secs]25210.868: [Tenured: 266146K->241318K(349568K), 1.2399896 secs]
339295K->241318K(519168K), 1.2888920 secs]
25212.108: [Full GC 25212.109: [Tenured[Unloading class
sun.reflect.GeneratedConstructorAccessor314]
[Unloading class sun.reflect.GeneratedMethodAccessor759]
[Unloading class sun.reflect.GeneratedConstructorAccessor315]
[Unloading class sun.reflect.GeneratedConstructorAccessor268]
[Unloading class sun.reflect.GeneratedConstructorAccessor269]
[Unloading class sun.reflect.GeneratedConstructorAccessor316]
[Unloading class sun.reflect.GeneratedConstructorAccessor313]
: 241318K->226997K(349568K), 1.5769614 secs] 241318K->226997K(519168K),
[Perm : 27510K->27498K(65536K)], 1.5771121 secs]

I still have below issues:

1.  Why do a partially scavenge while Young generation still has lot of
free memory?
Your guess is that the application may be requesting the allocation of a
large object, but I think it is impossible for our application to have
such a large obejct (about 169 - 86 = 83 M).
Is there any known issue about this kind of unusual gc?

And is the ratio setting between Young generation and Tenured generation
possible to improve it? Our current ratio is as default 1:2 (Young :
Tenured)

2. We could see that some unloading classes occur in the next full gc,
and seems JVM has some CR on this, is it possible related to this OOM?
or if it is related to application issue, how could we avoid it?

3. We have studied JVM doc on
http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html
See: 5.2.3 Out-of-Memory Exceptions
The throughput collector will throw an out-of-memory exception if too
much time is being spent doing garbage collection. For example, if the
JVM is spending more than 98% of the total time doing garbage collection
and is recovering less than 2% of the heap, it will throw an
out-of-memory expection. The implementation of this feature has changed
in 1.5. The policy is the same but there may be slight differences in
behavior due to the new implementation. 

And our gc log:
25213.699: [GC 25213.699: [DefNew: 756K->82K(169600K), 0.0078670
secs]25213.707: [Tenured[Unloading class
sun.reflect.GeneratedConstructorAccessor274]
: 226997K->221430K(349568K), 2.0173917 secs] 227753K->221430K(519168K),
2.0254397 secs]
25215.724: [Full GC 25215.724: [Tenured: 221430K->221422K(349568K),
1.2257687 secs] 221430K->221422K(519168K), [Perm :
27496K->27496K(65536K)], 1.2259138 secs]

Seems our gc log reveals a behavior like "if the JVM is spending more
than 98% of the total time doing garbage collection and is recovering
less than 2% of the heap", is this the time server threw the OOM?

Thanks.

Best wishes,
Steve Zhuang

-----Original Message-----
From: Y Srinivas Ramakrishna [mailto:Y.S.Ramakrishna at Sun.COM]
Sent: Wednesday, January 16, 2008 3:51 AM
To: CHRIS GUAN (ISDC-ISD-OOCLL/SHA)
Subject: Re: RE: Strange isssue about OOM

I forgot to send a pointer to the CR:-

http://bugs.sun.com/view_bug.do?bug_id=6206427

There really is not all that much to see in the bug report though except
that newer jvm's do this by default so you do not have to explicitly
enable the feature.

-- ramki

----- Original Message -----
From: Y Srinivas Ramakrishna <Y.S.Ramakrishna at Sun.COM>
Date: Tuesday, January 15, 2008 11:22 am
Subject: Re: RE: Strange isssue about OOM
To: chris.guan at oocl.com
Cc: hotspot-gc-use at openjdk.java.net

> Hi Chris --
> 
> > Thanks for your help.
> > 
> > Java version "1.4.2_13";
> > Oracle Application Server Containers for J2EE 10g (10.1.2.2.0) ; No 
> > System.gc(); WebService;
> > 
> > Would you please explain "HandlePromotionFailure"?Where to see CR 
> > 6206427.
> > 
> 
> In older versions of the JVM (such as the one you are using), we 
> didn't have a mechanism to recover from a partial scavenge if the 
> scavenge could not be completed because of lack of sufficient space in

> the old generation to promote tenured objects into. So we used to be 
> very conservative in starting a scavenge and probably conservative in 
> precipitating a full collection if we felt the next scavenge would not

> succeed. This conservatism could cause us to waste free space in the 
> old generation. See for example:-
> 
> http://java.sun.com/docs/hotspot/gc1.4.2/#4.4.2.%20Young%20Generation%
> 20Guarantee|outline
> 
> In later JVM's, we added the ability to recover from a partially 
> completed scavenge and to compact the entire heap. This allowed us to 
> be less pessimistic about going into a scavenge, and would waste less 
> space in the old generation even when you run with large young 
> generations. See:-
> 
> http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html#0.0.0.0.%20You
> ng%20Generation%20Guarantee%7Coutline
> 
> I'd actually recommend upgrading to a more recent JVM where you'd see 
> performance improvements, but if that is not possible, then with 
> 1.4.2_13, you should use -XX:+HandlePromotionFailure which should 
> allow you to make more efficient use of the heap by avoiding the issue

> described above.
> 
> As to the question of why you need to do, for example, the following 
> scavenge :-
> 
> > > 25313.039: [GC 25313.039: [DefNew: 2421K->91K(169600K), 0.0079322
> > > secs]25313.047: [Tenured: 220264K->220182K(349568K), 1.3464051 
> > > secs]
> 
> My only guess is that the application may be requesting the allocation

> of a large object. But that's a guess. (Yes, we should ideally have 
> that information be available from the gc logs.)
> 
> -- ramki
> 
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

IMPORTANT NOTICE
Email from OOCL is confidential and may be legally privileged.  If it is not intended for you, please delete it immediately unread.  The internet cannot guarantee that this communication is free of viruses, interception or interference and anyone who communicates with us by email is taken to accept the risks in doing so.  Without limitation, OOCL and its affiliates accept no liability whatsoever and howsoever arising in connection with the use of this email.  Under no circumstances shall this email constitute a binding agreement to carry or for provision of carriage services by OOCL, which is subject to the availability of carrier's equipment and vessels and the terms and conditions of OOCL's standard bill of lading which is also available at http://www.oocl.com.