Advice on finding out the cause of test failure of VarHandleTestAccessBoolean on windows x64

[Paul Sandoz]

Hi John,

Thanks.

I managed to boil this down to a reproducible test case just using Unsafe that fails on windows x64 but not on x86.

I think there is sufficient information to log a bug against HotSpot.

It can be reproduced by applying the webrev to jdk9/dev:

  http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8161444-vhs-bitwise-atomics/webrev/

compiling [2] with:

  javac --add-exports java.base/jdk.internal.misc=ALL-UNNAMED VarHandleTestAccessBoolean.java

then running [2] with [1].

The test case runs the following in a loop:

    static void testInstanceField(VarHandleTestAccessBoolean recv) {
        boolean o = UNSAFE.getAndBitwiseAndBoolean(recv, V_OFFSET, false);
        boolean r = UNSAFE.compareAndExchangeBooleanVolatile(recv, V_OFFSET, false, true);
    }

then runs the following once:

    static void testStaticFieldUnsafeBoolean() {
        static_v = true;

        boolean o = UNSAFE.getAndBitwiseAndBoolean(STATIC_V_BASE, STATIC_V_OFFSET, false);
        if (o != true) throw new RuntimeException();

        boolean x = static_v;
        if (x != (boolean)(true & false)) throw new RuntimeException();
    }

where the second test fails. Replace UNSAFE.getAndBitwiseAndBoolean with UNSAFE.getAndBitwiseAndByte and there is no failure:

    static void testStaticFieldUnsafeByte() {
        static_v = true;

        byte bo = UNSAFE.getAndBitwiseAndByte(STATIC_V_BASE, STATIC_V_OFFSET, (byte)0);
        boolean o = bo != 0;
        if (o != true) throw new RuntimeException();

        boolean x = static_v;
        if (x != (boolean)(true & false)) throw new RuntimeException();
    }

I can produce assembler output on request.

Thanks,
Paul.

[1]
#!/bin/sh

echo test 1
java -XX:+UnlockDiagnosticVMOptions \
   --add-exports java.base/jdk.internal.misc=ALL-UNNAMED \
   -XX:-TieredCompilation \
   VarHandleTestAccessBoolean


echo test 2
java -XX:+UnlockDiagnosticVMOptions \
   --add-exports java.base/jdk.internal.misc=ALL-UNNAMED \
   -XX:-TieredCompilation \
   -XX:DisableIntrinsic=_getByteVolatile \
   VarHandleTestAccessBoolean


echo test 3
java -XX:+UnlockDiagnosticVMOptions \
   --add-exports java.base/jdk.internal.misc=ALL-UNNAMED \
   -XX:-TieredCompilation \
   -XX:CompileCommand=dontinline,jdk.internal.misc.Unsafe::getAndBitwiseAndByte \
   VarHandleTestAccessBoolean


echo test 4
java -XX:+UnlockDiagnosticVMOptions \
   --add-exports java.base/jdk.internal.misc=ALL-UNNAMED \
   -Xcomp \
   VarHandleTestAccessBoolean


[2]
import java.lang.reflect.Field;

public class VarHandleTestAccessBoolean {

    static final jdk.internal.misc.Unsafe UNSAFE;


    static boolean static_v;
    static final Object STATIC_V_BASE;
    static final long STATIC_V_OFFSET;


    boolean v;
    static final long V_OFFSET;


    static {
        try {
            try {
                Field f = jdk.internal.misc.Unsafe.class.getDeclaredField("theUnsafe");
                f.setAccessible(true);
                UNSAFE = (jdk.internal.misc.Unsafe) f.get(null);
            } catch (Exception e) {
                throw new RuntimeException("Unable to get Unsafe instance.", e);
            }

            Field vField = VarHandleTestAccessBoolean.class.getDeclaredField("v");
            V_OFFSET = UNSAFE.objectFieldOffset(vField);

            Field staticVField = VarHandleTestAccessBoolean.class.getDeclaredField("static_v");
            STATIC_V_BASE = UNSAFE.staticFieldBase(staticVField);
            STATIC_V_OFFSET = UNSAFE.staticFieldOffset(staticVField);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    public static void main(String[] args) {
        VarHandleTestAccessBoolean t = new VarHandleTestAccessBoolean();
        for (int i = 0; i < 60_000; i++) {
            testInstanceField(t);
        }

        try {
            testStaticFieldUnsafeBoolean();
        } catch (Exception e) {
            System.err.println("FAILED: testStaticFieldUnsafeBoolean");
        }
        try {
            testStaticFieldUnsafeByte();
        } catch (Exception e) {
            System.err.println("FAILED: testStaticFieldUnsafeByte");
        }
    }

    static void testInstanceField(VarHandleTestAccessBoolean recv) {
        boolean o = UNSAFE.getAndBitwiseAndBoolean(recv, V_OFFSET, false);
        boolean r = UNSAFE.compareAndExchangeBooleanVolatile(recv, V_OFFSET, false, true);
    }

    static void testStaticFieldUnsafeBoolean() {
        static_v = true;

        boolean o = UNSAFE.getAndBitwiseAndBoolean(STATIC_V_BASE, STATIC_V_OFFSET, false);
        if (o != true) throw new RuntimeException();

        boolean x = static_v;
        if (x != (boolean)(true & false)) throw new RuntimeException();
    }

    static void testStaticFieldUnsafeByte() {
        static_v = true;

        byte bo = UNSAFE.getAndBitwiseAndByte(STATIC_V_BASE, STATIC_V_OFFSET, (byte)0);
        boolean o = bo != 0;
        if (o != true) throw new RuntimeException();

        boolean x = static_v;
        if (x != (boolean)(true & false)) throw new RuntimeException();
    }
}



> On 12 Aug 2016, at 13:18, John Rose <john.r.rose at oracle.com> wrote:
> 
> On Aug 12, 2016, at 8:36 AM, Paul Sandoz <paul.sandoz at oracle.com> wrote:
>> 
>> Hi,
>> 
>> Please can someone help/advice how to find the cause of a test failure i am observing.
> 
> It might be a failure of alias analysis.  The T_BOOLEAN unsafe access, when applied to
> a real boolean field of a class, can weaken to the same code as a getfield or putfield.
> A later getfield can "see" the most recent matching getfield or putfield, whether it is
> a bytecode or a successfully weakened unsafe access.
> 
> But the CAS address probably won't weaken to a field address, either because CAS
> is complicated (doesn't match a get/set bytecode semantics) or because T_BYTE doesn't
> match T_BOOLEAN.  In that case, maybe the JIT is pulling the previously read T_BOOLEAN
> access across the T_BYTE access, on the grounds that the T_BYTE access cannot alias.
> 
> If all this is true, we have a C2 issue related to mismatched loads and stores.
> 
> Possible test for hypothesis:  Obscure the offset portion of the boolean field, in the test
> code (run it through a volatile variable, or an all-constant array indexed by a random number).
> See if the problem persists.  Of course, you'll detune the optimization in that case also,
> which could mask the bug.  To do this you have to use raw unsafe, not a VH.  Could lead
> (at least) to a good minimal reproducer.
> 
> I can think of several reasons why the above may *not* be your problem, but it's worth a look.
> Maybe a kind C2 person can look into it?  (Someone who fixed the last store-mismatch bug? :-) )
> 
> VarHandles are throwing a lot of really new, cool code shapes into C2, which will lead to bugs
> like this.  It's the same kind of bug we'll also see with more aggressive use of pointer reshaping
> by Panama APIs.  (E.g., a "struct stat" stashed in a Java long[] in the heap.)
> 
> — John