DirectByteBuffer change proposal

[Peter Levart]

Hi Biju,

Just to clear any doubts. You have seen sleep() being called with JDK9 
which already includes the patch for 6857566 ?

The cleanup of native memory for DirecByteBuffers was and after 6857566 
still is designed around sun.misc.Cleaner which is basically a 
PhantomReference pointing to the DirecByteBuffer. When DirectByteBuffer 
instance becomes phantom-reachable, VM will at some later time discover 
it as a pending reference and ReferenceHandlerThread will "clean" it 
(execute Cleaner's thunk) releasing the native memory. So ordinarily, 
cleanup of native memory is performed asynchronously short time after VM 
decides to execute code to discover pending references. This is usually 
performed when heap allocation reaches certain point (maybe also 
periodically?). The problem is that memory pressure detected by VM is 
based on heap memory allocation which does not include native memory. It 
can and frequently happens that there's no heap memory pressure, but 
native memory reserved for DirectByteBuffers is exhausted.

Current strategy works by waiting until this exhaustion happens and then:
- attempts to "help" ReferenceHandlerThread to drain the pending 
references already discovered by VM, executing any sun.misc.Cleaners on 
the way; if that does not help, it
- triggers System.gc() which hopefully also triggers discovery of 
pending references, followed by a round of ReferenceHandlerThread 
helping; if that does not help, it
- retries the gc/tryHandlePending cycle, introducing exponentially 
increasing delays (from 1ms up to 512ms) which also acts as some kind of 
back-pressure; if that does not help, it
- considers native memory full and throws OutOfMemoryError

You propose to add triggering of gc() when native memory occupancy 
reaches certain % of max. memory allowed for native buffers. This sounds 
reasonable as it mimics roughly what happens when heap allocation 
reaches certain point. Reference handling would still be performed 
asynchronously solely by ReferenceHandlerThread in this case (no helping 
with tryHandlePending()) until native memory is exhausted. But note that 
System.gc() is not free. It usually is implemented as a blocking call 
which triggers safe-point VM processing and waits for it to complete 
before returning. So if you wanted to see low latencies for native 
buffer allocations, this should be performed by a background thread that 
continuously monitors current occupancy. You can try doing this in 
client code. There's a JMX bean that exposes native buffer allocation 
state (BufferPoolMXBean). Can you try doing this and report if it helps 
with allocation latencies in your application?

Regards, Peter

On 08/11/2015 11:43 PM, Biju G.S Nair wrote:
> Hi Peter,
>  Thanks for the background. Yes. We have seen sleep being called since 
> we are using DirectByteBuffers in GBs and the plan is to see whether 
> we can go up more than 90 GB. Also we have seen in certain version of 
> Linux kernel the 100 ms sleep was insufficient (looking for the real 
> issue on the kernel end). This is primarily to best use all the 
> resources on our servers which are mostly 32 cores and > 126 GB 
> physical memory. By giving users the additional option to set the 
> threshold that would give users like us additional lever to control 
> based on the work load/pattern. That is the thought. Hope this helps. 
> Please let me know if you have any further questions.
>
> Thanks,
> Biju
>
> On Tue, Aug 11, 2015 at 5:22 PM, Peter Levart <peter.levart at gmail.com 
> <mailto:peter.levart at gmail.com>> wrote:
>
>     Hi Biju,
>
>     When I was preparing this patch for JDK9, I did the following
>     measurement: Using LongAdders (to avoid Heisenberg) I counted
>     various exit paths from Bits.reserveMemory() during a test that
>     spawned 128 allocating threads on a 4-core i7 machine, allocating
>     direct buffers randomly sized between 256KB and 1MB for 60
>     seconds, using -XX:MaxDirectMemorySize=512m:
>
>     Total of 909960  allocations were performed:
>
>     - 247993 were satisfied before attempting to help ReferenceHandler
>     thread
>     - 660184 were satisfied while helping ReferenceHandler thread but
>     before triggering System.gc()
>     - 1783 were satisfied after triggering System.gc() but before
>     doing any sleep
>     - no sleeping has been performed
>
>     The same test, just changing -XX:MaxDirectMemorySize=128m (that
>     means 1MB per thread each allocating direct buffers randomly sized
>     between 256KB and 1MB):
>
>     Total of 579943 allocations were performed:
>
>     - 131547 were satisfied before attempting to help ReferenceHandler
>     thread
>     - 438345 were satisfied while helping ReferenceHandler thread but
>     before triggering System.gc()
>     - 10016 were satisfied after triggering System.gc() but before
>     doing any sleep
>     - 34 were satisfied after sleep(1)
>     - 1 was satisfied after sleep(1) followed by sleep(2)
>
>
>     Have your observations been different? Did you observe sleep()
>     been called in a real-world application?
>
>
>     Regards, Peter
>
>
>
>     On 08/11/2015 04:11 PM, Biju G.S Nair wrote:
>>     Hello All,
>>         While the patchhttps://bugs.openjdk.java.net/browse/JDK-6857566
>>     currently applied to jdk 9 (which I had requested to be back ported to JDK
>>     8 & 7) fixes the OOM exception during memory allocation by exponentially
>>     increasing the sleep time, this can negatively impact low latency
>>     applications using DirectByteBuffers. If we are able to provide a new JVM
>>     parameter which the users can set to a percentage value of DirectBuffer use
>>     as threshold when the "System.gc()" call in java/nio/Bits.java to be made,
>>     then the probability of sleep time being much lower is high. Also it gives
>>     users some control over when the gc() need to be requested instead of
>>     starting the gc() at the last moment when the direct memory is used fully.
>>     Without knowing all the details, to me it looks like a straight forward
>>     change. Let me know if there is any issue with the proposed change. If this
>>     change is a possibility let me know how I can make a request for this
>>     change.
>>
>>     Thanks,
>>     Biju
>
>