RFR: 8352773: JVMTI should disable events during java upcalls

[Chris Plummer]

On Tue, 25 Mar 2025 20:36:28 GMT, Chris Plummer <cjplummer at openjdk.org> wrote:

> Calling ThreadGroupReference.groups() from an event handler can cause a deadlock. Details in first comment. Tested with :jdk_lang on all supported platforms and tier1, tier2, tier3, and tier5 svc testing.

The reason is because this JDI API eventually ends up with JVMTI doing a java upcall to ThreadGroup.subgroupsAsArray(), which can trigger class loading the first time it is called. Thjis results in a ClassPrepareEvent that the debugger will get, but not process because its event handler thread is blocked on the ThreadGroupReference.groups(), so we have a deadlock.

The workaround is to make ThreadGroupReference.groups() not trigger any class loading. The fix is subtle. Before the fix, the java stack trace at the time of the ClasPrepareEvent looks like:

java.util.Arrays.copyOf(java.lang.Object[], int, java.lang.Class) bci:18 line:3513
java.util.ArrayList.toArray(java.lang.Object[]) bci:21 line:401
java.lang.ThreadGroup.subgroupsAsArray() bci:10 line:751

This is ThreadGroup.subgroupsAsArray:

private ThreadGroup[] subgroupsAsArray() {
    List<ThreadGroup> groups = synchronizedSubgroups();
    return groups.toArray(new ThreadGroup[0]);
}

This is ArrayList.toArray():

public <T> T[] toArray(T[] a) {
    if (a.length < size)
        // Make a new array of a's runtime type, but my contents:
        return (T[]) Arrays.copyOf(elementData, size, a.getClass());  <--- Line 401
    System.arraycopy(elementData, 0, a, 0, size);
    if (a.length > size)
        a[size] = null;
    return a;
}

And this is Arrays.copyOf():

public static <T,U> T[] copyOf(U[] original, int newLength, Class<? extends T[]> newType) {
    @SuppressWarnings("unchecked")
    T[] copy = ((Object)newType == (Object)Object[].class)
        ? (T[]) new Object[newLength]
        : (T[]) Array.newInstance(newType.getComponentType(), newLength);
    System.arraycopy(original, 0, copy, 0,
                     Math.min(original.length, newLength));
    return copy;
}

Line 3513 is actually the return statement, so this seems incorrect of by a bit. Adding some tracing of ClassPrepareEvents shows that we are currently handling the following:

cbClassPrepare: java.lang.reflect.Array

So it looks like we took the Array.newInstance() path, which triggered class loading of java.lang.reflect.Array. After the fix we never end up in Arrays.copyOf(). Instead ArrayList.toArray() calls System.arraycopy() directly, avoiding any class loading..

-------------

PR Comment: https://git.openjdk.org/jdk/pull/24236#issuecomment-2752702624