DirectByteBuffer change proposal

Wed Aug 12 14:05:49 UTC 2015

Hi Biju,

Well, seeing sleep() in JDK7 is understandable, since before the patch 
for 6857566, the strategy was to simply call System.gc() followed by a 
constant sleep(100L) every time native memory exhaustion was detected. 
There was no attempt to help process references synchronously before the 
1st sleep. My measurements after patch for 6857566 show that sleep() 
will only be called very infrequently in extreme situations provoked 
only with stress tests that do nothing but allocate direct buffers in 
multiple threads. I would be surprised if this happens in any real-world 
application. The amount of max. direct buffer memory should play a 
positive role here as more memory there is for native buffers, less 
probability it is that sleep() will be called. Sleep is called only in 
situations where the amount of native memory freed when calling 
gc()/tryHandlePending() - which should be all the memory allocated by 
currently unreachable buffers - is not enough to cope with concurrent 
allocation in a time slot between calling gc()/tryHandlePending() and 
attempting the allocation. At that point, sleep represents a kind of 
back-pressure to slow-down allocation and allow the program to continue 
to function over unexpected peaks (like DDOS attacks for example).

If you size the max. memory for native buffers to be big enough compared 
to max. expected allocated and used memory at any time with some spare 
share, sleep() will not occur. If it occurs regularly, it is a sign that 
you should increase the max. native memory limit, as your program really 
needs more memory.

Regards, Peter

On 08/12/2015 02:53 PM, Biju G.S Nair wrote:
> Thanks Peter for the detailed response. Currently we are in JDK 7 
> since the underlying datastore is still being certified for moving to 
> 8. So the issues we are seeing is not on JDK 9 with the patch. I have 
> requested the jdk8u-dev and jdk9u-dev team through the dev email 
> whether they can back port the patch to the two versions so that we 
> can test. Waiting to get the patch merged. Will update you as soon as 
> the JDK is updated.
>
> Thanks,
> Biju
>
> On Wed, Aug 12, 2015 at 2:51 AM, Peter Levart <peter.levart at gmail.com 
> <mailto:peter.levart at gmail.com>> wrote:
>
>     Hi Biju,
>
>     Just to clear any doubts. You have seen sleep() being called with
>     JDK9 which already includes the patch for 6857566 ?
>
>     The cleanup of native memory for DirecByteBuffers was and after 
>     6857566 still is designed around sun.misc.Cleaner which is
>     basically a PhantomReference pointing to the DirecByteBuffer. When
>     DirectByteBuffer instance becomes phantom-reachable, VM will at
>     some later time discover it as a pending reference and
>     ReferenceHandlerThread will "clean" it (execute Cleaner's thunk)
>     releasing the native memory. So ordinarily, cleanup of native
>     memory is performed asynchronously short time after VM decides to
>     execute code to discover pending references. This is usually
>     performed when heap allocation reaches certain point (maybe also
>     periodically?). The problem is that memory pressure detected by VM
>     is based on heap memory allocation which does not include native
>     memory. It can and frequently happens that there's no heap memory
>     pressure, but native memory reserved for DirectByteBuffers is
>     exhausted.
>
>     Current strategy works by waiting until this exhaustion happens
>     and then:
>     - attempts to "help" ReferenceHandlerThread to drain the pending
>     references already discovered by VM, executing any
>     sun.misc.Cleaners on the way; if that does not help, it
>     - triggers System.gc() which hopefully also triggers discovery of
>     pending references, followed by a round of ReferenceHandlerThread
>     helping; if that does not help, it
>     - retries the gc/tryHandlePending cycle, introducing exponentially
>     increasing delays (from 1ms up to 512ms) which also acts as some
>     kind of back-pressure; if that does not help, it
>     - considers native memory full and throws OutOfMemoryError
>
>     You propose to add triggering of gc() when native memory occupancy
>     reaches certain % of max. memory allowed for native buffers. This
>     sounds reasonable as it mimics roughly what happens when heap
>     allocation reaches certain point. Reference handling would still
>     be performed asynchronously solely by ReferenceHandlerThread in
>     this case (no helping with tryHandlePending()) until native memory
>     is exhausted. But note that System.gc() is not free. It usually is
>     implemented as a blocking call which triggers safe-point VM
>     processing and waits for it to complete before returning. So if
>     you wanted to see low latencies for native buffer allocations,
>     this should be performed by a background thread that continuously
>     monitors current occupancy. You can try doing this in client code.
>     There's a JMX bean that exposes native buffer allocation state
>     (BufferPoolMXBean). Can you try doing this and report if it helps
>     with allocation latencies in your application?
>
>     Regards, Peter
>
>
>     On 08/11/2015 11:43 PM, Biju G.S Nair wrote:
>>     Hi Peter,
>>        Thanks for the background. Yes. We have seen sleep being
>>     called since we are using DirectByteBuffers in GBs and the plan
>>     is to see whether we can go up more than 90 GB. Also we have seen
>>     in certain version of Linux kernel the 100 ms sleep was
>>     insufficient (looking for the real issue on the kernel end). This
>>     is primarily to best use all the resources on our servers which
>>     are mostly 32 cores and > 126 GB physical memory. By giving users
>>     the additional option to set the threshold that would give users
>>     like us additional lever to control based on the work
>>     load/pattern. That is the thought. Hope this helps. Please let me
>>     know if you have any further questions.
>>
>>     Thanks,
>>     Biju
>>
>>     On Tue, Aug 11, 2015 at 5:22 PM, Peter Levart
>>     <peter.levart at gmail.com <mailto:peter.levart at gmail.com>> wrote:
>>
>>         Hi Biju,
>>
>>         When I was preparing this patch for JDK9, I did the following
>>         measurement: Using LongAdders (to avoid Heisenberg) I counted
>>         various exit paths from Bits.reserveMemory() during a test
>>         that spawned 128 allocating threads on a 4-core i7 machine,
>>         allocating direct buffers randomly sized between 256KB and
>>         1MB for 60 seconds, using -XX:MaxDirectMemorySize=512m:
>>
>>         Total of 909960  allocations were performed:
>>
>>         - 247993 were satisfied before attempting to help
>>         ReferenceHandler thread
>>         - 660184 were satisfied while helping ReferenceHandler thread
>>         but before triggering System.gc()
>>         - 1783 were satisfied after triggering System.gc() but before
>>         doing any sleep
>>         - no sleeping has been performed
>>
>>         The same test, just changing -XX:MaxDirectMemorySize=128m
>>         (that means 1MB per thread each allocating direct buffers
>>         randomly sized between 256KB and 1MB):
>>
>>         Total of 579943 allocations were performed:
>>
>>         - 131547 were satisfied before attempting to help
>>         ReferenceHandler thread
>>         - 438345 were satisfied while helping ReferenceHandler thread
>>         but before triggering System.gc()
>>         - 10016 were satisfied after triggering System.gc() but
>>         before doing any sleep
>>         - 34 were satisfied after sleep(1)
>>         - 1 was satisfied after sleep(1) followed by sleep(2)
>>
>>
>>         Have your observations been different? Did you observe
>>         sleep() been called in a real-world application?
>>
>>
>>         Regards, Peter
>>
>>
>>
>>         On 08/11/2015 04:11 PM, Biju G.S Nair wrote:
>>>         Hello All,
>>>             While the patchhttps://bugs.openjdk.java.net/browse/JDK-6857566
>>>         currently applied to jdk 9 (which I had requested to be back ported to JDK
>>>         8 & 7) fixes the OOM exception during memory allocation by exponentially
>>>         increasing the sleep time, this can negatively impact low latency
>>>         applications using DirectByteBuffers. If we are able to provide a new JVM
>>>         parameter which the users can set to a percentage value of DirectBuffer use
>>>         as threshold when the "System.gc()" call in java/nio/Bits.java to be made,
>>>         then the probability of sleep time being much lower is high. Also it gives
>>>         users some control over when the gc() need to be requested instead of
>>>         starting the gc() at the last moment when the direct memory is used fully.
>>>         Without knowing all the details, to me it looks like a straight forward
>>>         change. Let me know if there is any issue with the proposed change. If this
>>>         change is a possibility let me know how I can make a request for this
>>>         change.
>>>
>>>         Thanks,
>>>         Biju
>>
>>
>
>