sun.nio.ch.Util: Don't cache an unlimited amount of memory

[Evan Jones]

Summary: nio Util caches an unlimited amount of memory for temporary direct
ByteBuffers, which effectively is a native memory leak. Applications that
do large I/Os can inadvertently waste gigabytes of native memory or run out
of memory. I suggest it should only cache a "small" amount of memory
per-thread (e.g. 1 MB), and maybe have a flag to allow changing the limit
for applications where this causes a performance regression.

1. Would JDK committers support this change?
2. If so, any suggestions for the default and/or how to override it with a
flag?

Tony Printezis (CCed here) added a flag to Twitter's internal JVM/JDK to
limit the size of this cache, which we could probably use that as a
starting point for a patch.


Additional Details:

At Twitter, we recently ran into a "native memory leak" caused
by sun.nio.ch.Util caching many gigabytes of native memory. Our program
makes the mistake of using heap ByteBuffers for I/O, and occasionally will
create a very large buffer and write it out a network socket. nio Util
caches a temporary direct ByteBuffer that is the maximum size a thread has
ever used. Thus, for a long lived thread, this memory usage will only ever
increase. This looks like a native memory leak, causing long lived
applications to continue to always use more and more native memory, until
it hits some limit and gets killed.

We aren't the only people who have this this problem. Norman Maurer
described running into this at Apple (some notes in this Twitter
discussion: https://twitter.com/TonyPrintezis/status/667117354438885376).
Ariel Weisberg described running into this as part of Cassandra:
https://twitter.com/ArielWeisberg/status/667138444561424384

My quick-and-dirty testing shows that removing the cache completely hurts
the performance of applications that use heap ByteBuffers. This is
particularly bad for "small" I/O, since the overhead of allocating and
deallocating memory is high, relative to the copy and the I/O. My
suggestion isl thread cache a maximum of 1 MB of direct ByteBuffer(s). This
means small I/Os will be the same as they are now, and large I/Os will have
a small amount of overhead. However, I would rather have a performance
problem than a native memory leak. Anyone who cares about performance
should be using direct ByteBuffers anyway.

Tony Printezis discusses this issue at 42:38 in the following video:
http://www.youtube.com/watch?v=M9o1LVfGp2A&t=42m38s

-- 
Evan Jones
http://evanjones.ca/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20151227/deb26608/attachment-0001.html>