[PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Paul Sandoz]

Hi Adam,

From reading the thread i cannot tell if this is part of a wider solution including some yet to be proposed HotSpot changes.

As is i would be resistant to adding such standalone internal wrapper methods to Unsafe that have no apparent benefit within the OpenJDK itself since it's a maintenance burden.

Can you determine if the calls to UNSAFE.freeMemory/allocateMemory come from a DBB by looking at the call stack frame above the unsafe call?

Thanks,
Paul.

> On Feb 14, 2018, at 3:32 AM, Adam Farley8 <adam.farley at uk.ibm.com> wrote:
> 
> Hi All,
> 
> Currently, diagnostic core files generated from OpenJDK seem to lump all 
> of the 
> native memory usages together, making it near-impossible for someone to 
> figure 
> out *what* is using all that memory in the event of a memory leak.
> 
> The OpenJ9 VM has a feature which allows it to track the allocation of 
> native
> memory for Direct Byte Buffers (DBBs), and to supply that information into 
> the
> cores when they are generated. This makes it a *lot* easier to find out 
> what is using
> all that native memory, making memory leak resolution less like some dark 
> art, and
> more like logical debugging.
> 
> To use this feature, there is a native method referenced in Unsafe.java. 
> To open
> up this feature so that any VM can make use of it, the java code below 
> sets the 
> stage for it. This change starts letting people call DBB-specific methods 
> when 
> allocating native memory, and getting into the habit of using it.
> 
> Thoughts?
> 
> Best Regards
> 
> Adam Farley
> 
> P.S. Code:
> 
> diff --git 
> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template 
> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
> --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
> +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
> @@ -85,7 +85,7 @@
>                 // Paranoia
>                 return;
>             }
> -            UNSAFE.freeMemory(address);
> +            UNSAFE.freeDBBMemory(address);
>             address = 0;
>             Bits.unreserveMemory(size, capacity);
>         }
> @@ -118,7 +118,7 @@
> 
>         long base = 0;
>         try {
> -            base = UNSAFE.allocateMemory(size);
> +            base = UNSAFE.allocateDBBMemory(size);
>         } catch (OutOfMemoryError x) {
>             Bits.unreserveMemory(size, cap);
>             throw x;
> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java 
> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
> @@ -632,6 +632,26 @@
>     }
> 
>     /**
> +     * Allocates a new block of native memory for DirectByteBuffers, of 
> the
> +     * given size in bytes.  The contents of the memory are 
> uninitialized;
> +     * they will generally be garbage.  The resulting native pointer will
> +     * never be zero, and will be aligned for all value types.  Dispose 
> of
> +     * this memory by calling {@link #freeDBBMemory} or resize it with
> +     * {@link #reallocateDBBMemory}.
> +     *
> +     * @throws RuntimeException if the size is negative or too large 
> +     *                          for the native size_t type
> +     *
> +     * @throws OutOfMemoryError if the allocation is refused by the 
> system
> +     *
> +     * @see #getByte(long)
> +     * @see #putByte(long, byte)
> +     */
> +    public long allocateDBBMemory(long bytes) {
> +        return allocateMemory(bytes);
> +    }
> + 
> +    /**
>      * Resizes a new block of native memory, to the given size in bytes. 
> The
>      * contents of the new block past the size of the old block are
>      * uninitialized; they will generally be garbage.  The resulting 
> native
> @@ -687,6 +707,27 @@
>     }
> 
>     /**
> +     * Resizes a new block of native memory for DirectByteBuffers, to the
> +     * given size in bytes.  The contents of the new block past the size 
> of
> +     * the old block are uninitialized; they will generally be garbage. 
> The
> +     * resulting native pointer will be zero if and only if the requested 
> size
> +     * is zero.  The resulting native pointer will be aligned for all 
> value
> +     * types.  Dispose of this memory by calling {@link #freeDBBMemory}, 
> or
> +     * resize it with {@link #reallocateDBBMemory}.  The address passed 
> to
> +     * this method may be null, in which case an allocation will be 
> performed.
> +     *
> +     * @throws RuntimeException if the size is negative or too large 
> +     *                          for the native size_t type
> +     *
> +     * @throws OutOfMemoryError if the allocation is refused by the 
> system
> +     *
> +     * @see #allocateDBBMemory
> +     */
> +    public long reallocateDBBMemory(long address, long bytes) {
> +        return reallocateMemory(address, bytes);
> +    }
> +
> +    /**
>      * Sets all bytes in a given block of memory to a fixed value
>      * (usually zero).
>      *
> @@ -918,6 +959,17 @@
>         checkPointer(null, address);
>     }
> 
> +    /**
> +     * Disposes of a block of native memory, as obtained from {@link
> +     * #allocateDBBMemory} or {@link #reallocateDBBMemory}.  The address 
> passed
> +     * to this method may be null, in which case no action is taken.
> +     *
> +     * @see #allocateDBBMemory
> +     */
> +    public void freeDBBMemory(long address) {
> +        freeMemory(address);
> +    }
> +
>     /// random queries
> 
>     /**
> 
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 
> 741598. 
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU