RFR (S) CR 6857566: (bf) DirectByteBuffer garbage creation can outpace reclamation

[Aleksey Shipilev]

Hi,

There is the issue that annoys me all the time:
  https://bugs.openjdk.java.net/browse/JDK-6857566

Please review the rework of DirectByteBuffer cleanup mechanics fixing
that issue. Since DBB uses sun.misc.Cleaner as the infrastructure to
clean up things, we focus there:
  http://cr.openjdk.java.net/~shade/6857566/webrev.00/

Brief explanation is there in Cleaner comments.

Testing:
  - jtreg: jdk/nio tests on Linux x86_64/fastdebug
  - original sample test from 6857566, reproduces in a few seconds
nicely with -XX:MaxDirectMemorySize=16M; more than 90 minutes run with
patched build yield no OOMEs!
  - microbenchmarking, see below

The microbenchmark tests allocates ByteBuffer.allocateDirect(N), with
N=1,10,100,1000 bytes:
  http://cr.openjdk.java.net/~shade/6857566/directbb/

On my 2x2 i5 Linux x86_64 laptop:

Before patch (1 thread):
 direct_1         834 +- 159 ns/op
 direct_10        888 +- 262 ns/op
 direct_100       969 +- 307 ns/op
 direct_1000     1618 +-  46 ns/op

Before patch (4 threads):
 direct_1       <OOME!>
 direct_10      <OOME!>
 direct_100     <OOME!>
 direct_1000    <OOME!>

Single threaded allocators are working fine, but this means nothing,
because more threads just plainly OOME.

After patch (1 thread):
 direct_1        1645 +- 68 ns/op
 direct_10       1623 +- 53 ns/op
 direct_100      1704 +- 44 ns/op
 direct_1000     3327 +- 43 ns/op

After patch (4 threads):
 direct_1        4673 +- 80    ns/op
 direct_10       4673 +- 114   ns/op
 direct_100      4771 +- 89    ns/op
 direct_1000    22310 +- 14390 ns/op

Thanks,
-Aleksey.