RFR: 8284877: Check type compatibility before looking up method from receiver's vtable [v2]

[David Holmes]

On Thu, 21 Apr 2022 06:04:27 GMT, Yi Yang <yyang at openjdk.org> wrote:

>> Hi, can I have a review for this enhancement? This patch adds type compatibility check before method lookup for robustness. In some internal cases, serialization framework may improperly generate an object of wrong type, which leads JVM randomly crashes during method resolution.
>> 
>> For example:
>> 
>> invokevirtual selected method: receiver-class:java.util.ArrayList, resolved-class:com.taobao.forest.domain.util.LongMapSupportArrayList, resolved_method:com.taobao.forest.domain.util.LongMapSupportArrayList.toMap()Ljava/util/Map;, selected_method:0x458, vtable_index:56#
>> 
>> The real type of receiver is ArrayList, while the resolved method is LongMapSupportArrayList.toMap. VM attempts to select method as if looking up from receiver's vtable via vtable index of resolved method(i.e. attempts to lookup `toMap()` from 
>>  ArrayList), an invalid method or incorrect method would be selected, thus causing some strange crashes.
>> 
>> I think it's reasonable to add a type compatibility check before method lookup. If such an incompatible call is found, JVM could throw an exception instead.
>
> Yi Yang has updated the pull request incrementally with one additional commit since the last revision:
> 
>   add jtreg test

Sorry but I am firmly against this "enhancement". Unsafe.putObject is clearly being used incorrectly and it clearly states:

/**
   * Stores a reference value into a given Java variable.
   * <p>
   * Unless the reference {@code x} being stored is either null
   * or matches the field type, the results are undefined.


The bug is in the library issuing the Unsafe call. The VM does not, and should not, guard against such misuses of Unsafe.

-------------

PR: https://git.openjdk.java.net/jdk/pull/8241