RFR: 8284877: Check type compatibility before looking up method from receiver's vtable

[Yi Yang]

On Wed, 20 Apr 2022 07:27:01 GMT, David Holmes <dholmes at openjdk.org> wrote:

> I admit I'm still not completely clear on the exact situation here in terms of what code is invoking what method with what receiver. If the bytecode is malformed then that should have been detected before now - as long as verification has not been disabled.

Okay, let me clarify more clearer:)

In the following code, the type of receiver longValues is LongMapSupportArrayList statically.(User-defined classes, contains a toMap) method.

public class DefaultStdCategoryPropertyDO extends DefaultFeatureSupportDO implements StdCategoryPropertyDO {
    ...
    private LongMapSupportArrayList<StdCategoryPropertyValueDO> longValues = new LongMapSupportArrayList("valueId");
    ...
    public Map<Long, ? extends StdCategoryPropertyValueDO> getValuesMap() {
        return this.longValues.toMap();
    }
}


Third-party serialization framework generates a ill-type longValues, i.e.

public class DefaultStdCategoryPropertyDO extends DefaultFeatureSupportDO implements StdCategoryPropertyDO {
    ...
    private ArrayList longValues;
    ...
    public Map<Long, ? extends StdCategoryPropertyValueDO> getValuesMap() {
        return this.longValues.toMap();
    }
}

Since there is no receiver type check in LinkResolver, trying to call ArrayList.toMap causes JVM crashes in many strange places. As why this call escapes many checks before LinkResolver, I admit I don't know.

-------------

PR: https://git.openjdk.java.net/jdk/pull/8241