Stability of lambda serialization

[Brian Goetz]

Yes, when this came up the first several times, the consensus of the EG 
was that we would build on the de-facto, practical constraint that 
people have learned to live with for years when dealing with 
serialization in the presence of inner classes: make sure the same 
classfile bits are on both sides of the pipe.

Inner classes have a naming instability, where you have a stable method 
name but an unstable class name.  Lambdas have the opposite, but 
effectively equivalent instability: a stable class name but an unstable 
method name.  People have learned to use serialization with inner 
classes by the technique of keeping the classfile bits the same on both 
sides of the wire.  The exact same technique enables serialization of 
lambdas.

In particular, the EG rejected the extremes of "make it perfect" and 
"don't allow it at all" as being "the perfect is the enemy of the good." 
  Inner classes have been exactly this hostile to serialization for 15 
years, but people who want to use serialization have learned coping 
techniques to do so.  The compromises in the middle are squishy but 
better than either extreme.

Of course, if we can make it better without exposing additional 
complexity, so much the better.  For example, we did a few rounds of 
this by encoding the hash of the method name and signature so that 
simply reordering the methods in a class did not trigger such 
instabilities.  But the goal was never perfect stability; it was just 
"at least as good as inner classes."  I believe we've reached that goal.

Do you have a concrete proposal for how we would improve lambda 
serialization to be robust in the face of changes in order of capture? 
On the surface it looks entirely possible.

On 7/27/2013 12:52 PM, Sam Pullara wrote:
> I think the requirement should be that you have the same classes on each
> side of the wire.
>
> Sam
>
>
> On Tue, Jul 23, 2013 at 10:35 AM, Scott Stark <sstark at redhat.com
> <mailto:sstark at redhat.com>> wrote:
>
>     Red Hat has a concern regarding how fragile the default
>     serialization behavior of lambda expressions is in the current
>     reference implementation, currently:
>     ironmaiden:OpenJDK starksm$
>     /Library/Java/JavaVirtualMachines/jdk1.8.0.jdk/Contents/Home/bin/java -version
>     java version "1.8.0-ea"
>     Java(TM) SE Runtime Environment (build 1.8.0-ea-b98)
>     Java HotSpot(TM) 64-Bit Server VM (build 25.0-b40, mixed mode)
>
>     The problem is that the serialized form of a lambda expression
>     depends on the order in which captured arguments are declared. The
>     attached simple example demonstrates how easy it is for a trivial
>     reordering of the lambda code block to result in an inability to
>     deserialize a previously saved expression.
>
>     To produce this exception:
>     1. Run the serialization.AuthenticationContext.testWriteLambda
>     method with the lambda expression written as:
>            Authenticator a = (Authenticator & Serializable) (String
>     principal, char[] pass) -> {
>                // Run with p declared first when writing out the
>     /tmp/testWriteLambda.bin, then switch
>                // to declare u first when running testReadLambda
>                String p = "-> Password " + password + " <-";
>                String u = "-> User " + user + " <-";
>                return u + " " + p;
>            };
>     2. Change the lambda expression to:
>            Authenticator a = (Authenticator & Serializable) (String
>     principal, char[] pass) -> {
>                // Run with p declared first when writing out the
>     /tmp/testWriteLambda.bin, then switch
>                // to declare u first when running testReadLambda
>                String u = "-> User " + user + " <-";
>                String p = "-> Password " + password + " <-";
>                return u + " " + p;
>            };
>
>     Recompile and run serialization.AuthenticationContext.testReadLambda
>     to produce:
>
>     java.io.IOException: unexpected exception type
>              at
>     java.io.ObjectStreamClass.throwMiscException(ObjectStreamClass.java:1538)
>              at
>     java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1110)
>              at
>     java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1807)
>              at
>     java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
>              at
>     java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
>              at
>     serialization.AuthenticationContext.testReadLambda(AuthenticationContext.java:34)
>     ...
>     Caused by: java.lang.reflect.InvocationTargetException
>              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>              at
>     sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>              at
>     sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>              at
>     java.lang.invoke.SerializedLambda.readResolve(SerializedLambda.java:222)
>              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>              at
>     sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>              at
>     sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>              at
>     java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1104)
>              ... 30 more
>     Caused by: java.lang.IllegalArgumentException: Invalid lambda
>     deserialization
>              at
>     serialization.AuthenticationContext.$deserializeLambda$(AuthenticationContext.java:1)
>              ... 40 more
>
>     One does not see the same level of sensitivity to the ordering of
>     the serialization fields in a POJO as demonstrated by the
>     serialization.AuthenticationContext.testWritePOJO/testReadPOJO cases
>     where one can reorder the TestPOJO.{user,password} fields without
>     having serialization fail.
>
>     We would like to see at least that level of stability of the
>     serialized form of lambda expressions.
>
>