Bug in the Direct$Type$Buffer$RW$(int cap) constructor in src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template if the allocation of the cleaner fails

[John Platts]

I have discovered a bug in the Direct$Type$Buffer$RW$(int cap) constructor if the allocation of the cleaner fails as memory will be leaked if an Exception or Error is thrown by the allocation of the Deallocator or the creation of the cleaner.

Here is how the allocation bug in the Direct$Type$Buffer$RW$(int cap) constructor can be fixed:
    private static class Deallocator
        implements Runnable
    {
        private final long address;
        private final long size;
        private final int capacity;
        private int deallocationPerformed;

        private static final long DEALLOCATION_PERFORMED =
            UNSAFE.objectFieldOffset(Deallocator.class, "deallocationPerformed");

        private Deallocator(long address, long size, int capacity) {
            assert (address != 0);
            this.address = address;
            this.size = size;
            this.capacity = capacity;
        }

        public void run() {
            if(UNSAFE.getAndSetIntAcquire(this, DEALLOCATION_PERFORMED, 1) == 0) {
                UNSAFE.freeMemory(address);
                Bits.unreserveMemory(size, capacity);
            }
        }
    }

    // Primary constructor
    //
    Direct$Type$Buffer$RW$(int cap) {                   // package-private
#if[rw]
        super(-1, 0, cap, cap);
        boolean pa = VM.isDirectMemoryPageAligned();
        int ps = Bits.pageSize();
        long size = Math.max(1L, (long)cap + (pa ? ps : 0));
        Bits.reserveMemory(size, cap);

        long base = 0;
        Deallocator deallocator = null;
        try {
            base = UNSAFE.allocateMemory(size);
            UNSAFE.setMemory(base, size, (byte) 0);

            if (pa && (base % ps != 0)) {
                // Round up to page boundary
                address = base + ps - (base & (ps - 1));
            } else {
                address = base;
            }

            deallocator = new Deallocator(base, size, cap);
            cleaner = Cleaner.create(this, deallocator);
        } catch(Throwable ex) {
            if(deallocator != null) {
                deallocator.run();
            } else {
                if(base != 0)
                    UNSAFE.freeMemory(base);

                Bits.unreserveMemory(size, capacity);
            }

            throw ex;
        }
        att = null;
#else[rw]
        super(cap);
        this.isReadOnly = true;
#end[rw]
    }

The updated version of the Direct$Type$Buffer$RW$(int cap) above will ensure that the memory that was allocated by UNSAFE.allocateMemory(size) will get freed if the creation of the cleaner fails.

A deallocationPerformed field is also added to the Deallocator class to make sure that the memory that was allocated by the Direct$Type$Buffer$RW$(int cap) is freed exactly once. A UNSAFE.getAndSetIntAcquire is used to set deallocationPerformed to 1 in the Deallocator.run() method to make sure that the deallocation of memory by Deallocator.run is performed only once.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/nio-dev/attachments/20221003/518e2701/attachment.htm>