G1: SoftReference, 0 refs, 31.0027203 secs

[charlie hunt]

Hi Simone,

Seems very peculiar to see 0 SoftReferences processed and an incredibly high reported time.

Couple questions popped in my mind as I looked through the logs.

I’m assuming this is on Linux?  If so, could you confirm THP (transparent huge pages) is disabled?

And, did you happen to try tuning -XXSoftRefLRUPolicyMSPerMB smaller from the default of 1000 to say something as low as 1 to see if what you’re seeing goes away ?

Perhaps someone on the GC team has some thoughts as a situation where we would see 0 SoftReferences processed yet a high amount of time spent there.

thanks,

charlie

> On Jul 16, 2015, at 3:16 AM, Simone Bordet <simone.bordet at gmail.com> wrote:
> 
> Hi,
> 
> I have an application that reported very large (around 30 s) times to
> process *zero* SoftReferences, for example:
> 
> 6015.665: [SoftReference, 0 refs, 23.0169525 secs]6038.682:
> [WeakReference, 1 refs, 0.0046033 secs]6038.687: [FinalReference,
> 31647 refs, 0.0090301 secs]6038.696: [PhantomReference, 241 refs,
> 0.0048419 secs]6038.701: [JNI Weak Reference, 0.0000463 secs],
> 23.2166772 secs]
> 
> We have been hit by this anomaly a few times now, and in the attached
> logs (that also show the command line flags) it happened 3 times: at
> uptimes 6015.512, 6074.487, 6141.161.
> 
> What happens after these long pauses is that G1 goes into "GC overhead
> mode", tries to expand the heap (which fails because it's already
> expanded), but keeps the Eden at a very small size, resulting in a
> series of back-to-back collections that lasted almost 3 minutes where
> the MMU dropped making the application almost unusable.
> After that, G1 was able to recover to normal behavior.
> 
> I was wondering if anyone knows a little more about this issue (long
> times to process zero soft references), or whether it has been fixed
> in more recent releases.
> 
> We are not aware of any other process that could have caused this such
> as busy disk I/O or swapping (the machine has plenty of memory left
> and it is dedicated to the JVM), but we'll run jHiccup next time.
> However, the fact that it always happened during the processing of
> soft references seems suspicious.
> 
> Should I file an issue ?
> 
> Thanks !
> 
> -- 
> Simone Bordet
> http://bordet.blogspot.com
> ---
> Finally, no matter how good the architecture and design are,
> to deliver bug-free software with optimal performance and reliability,
> the implementation technique must be flawless.   Victoria Livschitz
> <akiba-20150713-gc.log.gz>_______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use