OOM without a full GC with G1GC

[Per Liden]

Hi,

On 2017-05-08 19:28, nezih yigitbasi wrote:
> Hi Per,
> I was also thinking about bumping up GCLockerRetryAllocationCount. But,
> the default value of this parameter is small (2), is that because having
> a large value for that can have other negative side effects? In other
> words, how safe is it to increase this number?

If a GC is needed to satisfy an allocation, that GC will be delayed if 
the GC locker is active. If multiple threads tries to allocate in this 
situation they will all be delayed until the GC has completed. However, 
there's currently no priority queue ensuring that allocations will be 
satisfied in some fair order (like first come, first served), once the 
GC locker becomes inactive. Instead waiting threads will compete for 
memory once the GC has completed. That can cause starvation if the same 
thread looses that race over and over again. 
GCLockerRetryAllocationCount basically dictates how many times a thread 
should retry its allocation in this situation. Setting it to a large 
number, will help avoid giving up too early and throw OOM. The downside 
is that an allocation can take longer to complete, but that's often 
better than a premature OOM.

cheers,
Per

>
> Thanks,
> Nezih
>
> 2017-05-08 2:49 GMT-07:00 Roman Kennke <rkennke at redhat.com
> <mailto:rkennke at redhat.com>>:
>
>     Am 08.05.2017 um 09:51 schrieb Per Liden:
>     > Hi,
>     >
>     > On 2017-05-06 00:12, nezih yigitbasi wrote:
>     >> This JVM runs the Presto distributed SQL engine and some threads use
>     >> native codecs to read from compressed files (I see from our logs that
>     >> GZIP is being used). Is it possible that JNI threads delay the full GC
>     >> and the JVM hits the GCLockerRetryAllocationCount?
>     >
>     > Yes, heavy use of JNI GetPrimitiveArrayCritical (used by libzip) can
>     > cause allocation starvation and premature OOM, which might be what
>     > you're seeing here. Try set GCLockerRetryAllocationCount to a big
>     > number. That would at least help avoid the premature OOM.
>
>     You might want to give Shenandoah GC a try. Not only is it specifically
>     targeted at low pause times with large heaps, it also does not hold back
>     GC because of JNI critical regions..
>
>     https://wiki.openjdk.java.net/display/shenandoah/Main
>     <https://wiki.openjdk.java.net/display/shenandoah/Main>
>
>     Best regards, Roman
>
>