RFR: JDK-8068721 - RMI-IIOP communication fails when ConcurrentHashMap is passed to remote method

[Mark Sheppard]

Hi Peter,
     I'll remove the synchronization ... I think your right there should 
only be a single thread of control executing
in a particular instance . Based on your prompt I ran a concurrent 
invocation test on the same remote reference and didn't
observe any conflict - separate IIOPOutputStream were used.

regards
Mark

On 24/03/2015 15:46, Peter Levart wrote:
> On 03/23/2015 10:25 PM, Mark Sheppard wrote:
>> Hi
>>  please oblige and review the following fix
>>
>> http://cr.openjdk.java.net/~msheppar/8068721/jdk9/corba/webrev/
>> http://cr.openjdk.java.net/~msheppar/8068721/jdk9/jdk/webrev/
>
> Hi Mark,
>
> Can an instance of ObjectStreamHook (ObjectOutputStream) be invoked 
> from multiple threads concurrently? I don't see any synchronization 
> elsewhere in ObjectStreamHook or ObjectOutputStream. Perhaps the 
> synchronization in putFields()/writeFields() is not needed.
>
> Also, in ObjecyOutputStream, writeFields() looks like this:
>
>     public void writeFields() throws IOException {
>         if (curPut == null) {
>             throw new NotActiveException("no current PutField object");
>         }
>         bout.setBlockDataMode(false);
>         curPut.writeFields();
>         bout.setBlockDataMode(true);
>     }
>
>
> ...meaning that NotActiveException is thrown in case user calls this 
> method before invoking putFields() which initializes the PutField 
> object. Should ObjectStreamHook behave the same?
>
> Regards, Peter
>
>>
>>
>> which addresses the issue in
>> https://bugs.openjdk.java.net/browse/JDK-8068721
>>
>> This relates to RMI-IIOP and the interplay between custom marshalling 
>> of ValueTypes and
>> the corresponding serialization of a Java class, in this case 
>> ConcurrentHashMap.
>>
>> ConcurrentHashMap changed its structure in jdk8 from that used in jdk7.
>> This resulted in modification to the readObject and writeObject methods,
>> and in particular, former serial fields were removed, resulting in
>> writeObject using PutField and readObject using defaultReadObject.
>> The writeObject invokes the putFields method of an ObjectOutputStream 
>> multiple times, and assumes
>> that it will receive the same PutField object instance for each 
>> invocation. The spec
>> doesn't endorse this behaviour - but that's what the implementation 
>> of ObjectOutputStream
>> provides. However in the case of RMI-IIOP, the OutputStreamHook, a 
>> subclass of ObjectOutputStream, returns a new instance for each
>> putFields invocation. Thus, the ConcurrentHashMap writeObject results 
>> in improper serialization in the context
>> of RMI-IIOP.
>> In the unmarshalling flow of ConcurrentHashMap, the readObject now 
>> uses defaultReadObject rather than GetField
>> In this call flow the IIOPInputStream attempts to ignore any 
>> primitive field, in the serialized data, that doesn't
>> correspond with a reflective field of the object. However, it leaves 
>> the primitive in the stream.
>> Thus, in the case of ConcurrentHashMap, which has serialized two 
>> integers and
>> an array of Segments (subclass of ReentrantLock), this results in 
>> erroneous
>> deserialization, with a misinterpretation of a value tag in the 
>> stream as an array length
>> and an attempt to allocate a very large array ensues, with an 
>> exception being thrown.
>>
>> The OutputStreamHook now returns the same instance of PutField 
>> allocated for each separate call of putFields method.
>> This highlights a need to tighten up and disambiguate the 
>> OutputObjectStream spec for putFields.
>>
>> IIOPInputStream now removes the primitive values from the stream.
>>
>> regards
>> Mark
>