RFR: 8306461: ObjectInputStream::readObject() should handle negative array sizes without throwing NegativeArraySizeExceptions

[Joe Darcy]

On Wed, 19 Apr 2023 17:04:07 GMT, Joe Darcy <darcy at openjdk.org> wrote:

>> This issue was reported by: Yakov Shafranovich ([yakovsh at amazon.com](mailto:yakovsh at amazon.com))
>> 
>> Currently, `ObjectInputStream::readObject()` doesn't explicitly checks for a negative array length in the deserialization stream. Instead it calls `j.l.r.Array::newInstance(..)` with the negative length which results in a `NegativeArraySizeException`. NegativeArraySizeException is an unchecked exception which is neither declared in the signature of `ObjectInputStream::readObject()` nor mentioned in its API specification. It is therefore not obvious for users of `ObjectInputStream::readObject()` that they may have to handle `NegativeArraySizeException`s. It would therefor be better if a negative array length in the deserialization stream would be automatically wrapped in an `InvalidClassException` which is a checked exception (derived from `IOException` via `ObjectStreamException`) and declared in the signature of `ObjectInputStream::readObject()`.
>> 
>> If we do the negative array length check in `ObjectInputStream::readObject()` before filtering, this will then also fix `ObjectInputFilter.FilterInfo::arrayLength()` which is defined as:
>> 
>> Returns:
>> the non-negative number of array elements when deserializing an array of the class, otherwise -1
>> 
>> but currently returns a negative value if the array length is negative.
>
> Please file a CSR for the proposed behavioral change.

> Hi @jddarcy,
> 
> I'm happy to create a CSR for this change, but I'm a little bit unsure about the details. From my understanding this qualifies as a behavioral change, right? But this behavior wasn't specified before at all. Neither did the API specification of `ObjectInputStream::readObject()` mention that it can throw a `NegativeArraySizeException` nor did the [Serialization Specification](https://docs.oracle.com/en/java/javase/20/docs/specs/serialization/index.html) mentioned the case of a negative array size.
> 
> Previously, `readObject()` could throw a `NegativeArraySizeException` which will now, with this PR, be changed into a `InvalidClassException`. Do you agree that this is an implementation detail and therefor the CSR should have "Implementation" Scope?
> 
> Thanks, Volker

Hi Volker,

Yes, the behavior wasn't specified, but that doesn't imply users haven't become reliant on it, hence the (behavioral) compatibility review via a CSR of the implementation change.

HTH

-------------

PR Comment: https://git.openjdk.org/jdk/pull/13540#issuecomment-1516545807