RFR: JDK-8068721 - RMI-IIOP communication fails when ConcurrentHashMap is passed to remote method

[Martin Buchholz]

Let us know if the serialization code of the collection classes can be
improved.

On Mon, Mar 23, 2015 at 2:25 PM, Mark Sheppard <mark.sheppard at oracle.com>
wrote:

> Hi
>  please oblige and review the following fix
>
> http://cr.openjdk.java.net/~msheppar/8068721/jdk9/corba/webrev/
> http://cr.openjdk.java.net/~msheppar/8068721/jdk9/jdk/webrev/
>
>
> which addresses the issue in
> https://bugs.openjdk.java.net/browse/JDK-8068721
>
> This relates to RMI-IIOP and the interplay between custom marshalling of
> ValueTypes and
> the corresponding serialization of a Java class, in this case
> ConcurrentHashMap.
>
> ConcurrentHashMap changed its structure in jdk8 from that used in jdk7.
> This resulted in modification to the readObject and writeObject methods,
> and in particular, former serial fields were removed, resulting in
> writeObject using PutField and readObject using defaultReadObject.
> The writeObject invokes the putFields method of an ObjectOutputStream
> multiple times, and assumes
> that it will receive the same PutField object instance for each
> invocation. The spec
> doesn't endorse this behaviour - but that's what the implementation of
> ObjectOutputStream
> provides. However in the case of RMI-IIOP, the OutputStreamHook, a
> subclass of ObjectOutputStream, returns a new instance for each
> putFields invocation. Thus, the ConcurrentHashMap writeObject results in
> improper serialization in the context
> of RMI-IIOP.
> In the unmarshalling flow of ConcurrentHashMap, the readObject now uses
> defaultReadObject rather than GetField
> In this call flow the IIOPInputStream attempts to ignore any primitive
> field, in the serialized data, that doesn't
> correspond with a reflective field of the object. However, it leaves the
> primitive in the stream.
> Thus, in the case of ConcurrentHashMap, which has serialized two integers
> and
> an array of Segments (subclass of ReentrantLock), this results in erroneous
> deserialization, with a misinterpretation of a value tag in the stream as
> an array length
> and an attempt to allocate a very large array ensues, with an exception
> being thrown.
>
> The OutputStreamHook now returns the same instance of PutField allocated
> for each separate call of putFields method.
> This highlights a need to tighten up and disambiguate the
> OutputObjectStream spec for putFields.
>
> IIOPInputStream now removes the primitive values from the stream.
>
> regards
> Mark
>