RFR: JDK-8068721 - RMI-IIOP communication fails when ConcurrentHashMap is passed to remote method

[Chris Hegarty]

On 23 Mar 2015, at 21:25, Mark Sheppard <mark.sheppard at oracle.com> wrote:

> ….
> 
> ConcurrentHashMap changed its structure in jdk8 from that used in jdk7.
> This resulted in modification to the readObject and writeObject methods,
> and in particular, former serial fields were removed, resulting in

The non-transient fields were removed, but the on-the-wire serial form stayed the same. Unfortunately, serialPersistentFields is missing the description of its elements, so the serial form does not show up in the javadoc. I sent suggested changes to address this in a separate mail.

> writeObject using PutField and readObject using defaultReadObject.
> The writeObject invokes the putFields method of an ObjectOutputStream multiple times, and assumes
> that it will receive the same PutField object instance for each invocation. The spec
> doesn't endorse this behaviour - but that's what the implementation of ObjectOutputStream
> provides.

I filed the following bug to tighten up the OOS.putFields spec:
  https://bugs.openjdk.java.net/browse/JDK-8075802

> However in the case of RMI-IIOP, the OutputStreamHook, a subclass of ObjectOutputStream, returns a new instance for each
> putFields invocation. Thus, the ConcurrentHashMap writeObject results in improper serialization in the context
> of RMI-IIOP.
> In the unmarshalling flow of ConcurrentHashMap, the readObject now uses defaultReadObject rather than GetField
> In this call flow the IIOPInputStream attempts to ignore any primitive field, in the serialized data, that doesn't
> correspond with a reflective field of the object. However, it leaves the primitive in the stream.

Ouch! This looks like a nasty bug in IIOPInputStream.

> Thus, in the case of ConcurrentHashMap, which has serialized two integers and
> an array of Segments (subclass of ReentrantLock), this results in erroneous
> deserialization, with a misinterpretation of a value tag in the stream as an array length
> and an attempt to allocate a very large array ensues, with an exception being thrown.
> 
> The OutputStreamHook now returns the same instance of PutField allocated for each separate call of putFields method.

This seems reasonable to me.

> This highlights a need to tighten up and disambiguate the OutputObjectStream spec for putFields.

Yes, it does. We should address it in 8075802.

I see you have changed OutputStreamHook to return the same instance for multiple calls, so adding a spec clarification, to that affect, to OOS.putFields() would not conflict with your change.

> IIOPInputStream now removes the primitive values from the stream.

I just looked at the code changes in the corba repo, and they look ok to me.

-Chris.

> regards
> Mark