Stability of lambda serialization

[David M. Lloyd]

As before, I continue to strongly disagree with this so-called "de 
facto" constraint.  In reality, we see real users who rely heavily on 
the *real* constraints which are outlined in the specification of 
serialization.  I think that real-world users are going to encounter 
bizarre issues at best, and security vulnerabilities at worst, due to 
the behavior of capture.

As I always do when this particular truism is brought up, I will counter 
your "the perfect is the enemy of the good" with "the good is the enemy 
of the correct".  A clever saying is never enough to justify incorrect 
behavior.  I think it's perfectly reasonable to capture "this", at most, 
but beyond that I think there is no benefit to having capture, at all, 
in the face of its instability.

I think the "equivalent stability" argument is also a bit fallacious. 
It's saying "anonymous classes are bad, so that's a free pass to do this 
similarly badly as well, in whatever way we want".  To be "as good as 
anonymous classes", we'd have to have capture which is stable by name at 
least.

But I personally I think this criteria is arbitrary and unnecessary in 
any case.  I think that we can easily guarantee stability of both 
captured variables and of method resolution by only allowing 
serialization of named method references and forbidding capture (other 
than "this").

On 07/27/2013 12:18 PM, Brian Goetz wrote:
> Yes, when this came up the first several times, the consensus of the EG
> was that we would build on the de-facto, practical constraint that
> people have learned to live with for years when dealing with
> serialization in the presence of inner classes: make sure the same
> classfile bits are on both sides of the pipe.
>
> Inner classes have a naming instability, where you have a stable method
> name but an unstable class name.  Lambdas have the opposite, but
> effectively equivalent instability: a stable class name but an unstable
> method name.  People have learned to use serialization with inner
> classes by the technique of keeping the classfile bits the same on both
> sides of the wire.  The exact same technique enables serialization of
> lambdas.
>
> In particular, the EG rejected the extremes of "make it perfect" and
> "don't allow it at all" as being "the perfect is the enemy of the good."
>   Inner classes have been exactly this hostile to serialization for 15
> years, but people who want to use serialization have learned coping
> techniques to do so.  The compromises in the middle are squishy but
> better than either extreme.
>
> Of course, if we can make it better without exposing additional
> complexity, so much the better.  For example, we did a few rounds of
> this by encoding the hash of the method name and signature so that
> simply reordering the methods in a class did not trigger such
> instabilities.  But the goal was never perfect stability; it was just
> "at least as good as inner classes."  I believe we've reached that goal.
>
> Do you have a concrete proposal for how we would improve lambda
> serialization to be robust in the face of changes in order of capture?
> On the surface it looks entirely possible.
>
> On 7/27/2013 12:52 PM, Sam Pullara wrote:
>> I think the requirement should be that you have the same classes on each
>> side of the wire.
>>
>> Sam
>>
>>
>> On Tue, Jul 23, 2013 at 10:35 AM, Scott Stark <sstark at redhat.com
>> <mailto:sstark at redhat.com>> wrote:
>>
>>     Red Hat has a concern regarding how fragile the default
>>     serialization behavior of lambda expressions is in the current
>>     reference implementation, currently:
>>     ironmaiden:OpenJDK starksm$
>>
>> /Library/Java/JavaVirtualMachines/jdk1.8.0.jdk/Contents/Home/bin/java
>> -version
>>     java version "1.8.0-ea"
>>     Java(TM) SE Runtime Environment (build 1.8.0-ea-b98)
>>     Java HotSpot(TM) 64-Bit Server VM (build 25.0-b40, mixed mode)
>>
>>     The problem is that the serialized form of a lambda expression
>>     depends on the order in which captured arguments are declared. The
>>     attached simple example demonstrates how easy it is for a trivial
>>     reordering of the lambda code block to result in an inability to
>>     deserialize a previously saved expression.
>>
>>     To produce this exception:
>>     1. Run the serialization.AuthenticationContext.testWriteLambda
>>     method with the lambda expression written as:
>>            Authenticator a = (Authenticator & Serializable) (String
>>     principal, char[] pass) -> {
>>                // Run with p declared first when writing out the
>>     /tmp/testWriteLambda.bin, then switch
>>                // to declare u first when running testReadLambda
>>                String p = "-> Password " + password + " <-";
>>                String u = "-> User " + user + " <-";
>>                return u + " " + p;
>>            };
>>     2. Change the lambda expression to:
>>            Authenticator a = (Authenticator & Serializable) (String
>>     principal, char[] pass) -> {
>>                // Run with p declared first when writing out the
>>     /tmp/testWriteLambda.bin, then switch
>>                // to declare u first when running testReadLambda
>>                String u = "-> User " + user + " <-";
>>                String p = "-> Password " + password + " <-";
>>                return u + " " + p;
>>            };
>>
>>     Recompile and run serialization.AuthenticationContext.testReadLambda
>>     to produce:
>>
>>     java.io.IOException: unexpected exception type
>>              at
>>
>> java.io.ObjectStreamClass.throwMiscException(ObjectStreamClass.java:1538)
>>              at
>>
>> java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1110)
>>              at
>>
>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1807)
>>              at
>>     java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
>>              at
>>     java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
>>              at
>>
>> serialization.AuthenticationContext.testReadLambda(AuthenticationContext.java:34)
>>
>>     ...
>>     Caused by: java.lang.reflect.InvocationTargetException
>>              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> Method)
>>              at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>
>>              at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>>              at
>>
>> java.lang.invoke.SerializedLambda.readResolve(SerializedLambda.java:222)
>>              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> Method)
>>              at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>
>>              at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>>              at
>>
>> java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1104)
>>              ... 30 more
>>     Caused by: java.lang.IllegalArgumentException: Invalid lambda
>>     deserialization
>>              at
>>
>> serialization.AuthenticationContext.$deserializeLambda$(AuthenticationContext.java:1)
>>
>>              ... 40 more
>>
>>     One does not see the same level of sensitivity to the ordering of
>>     the serialization fields in a POJO as demonstrated by the
>>     serialization.AuthenticationContext.testWritePOJO/testReadPOJO cases
>>     where one can reorder the TestPOJO.{user,password} fields without
>>     having serialization fail.
>>
>>     We would like to see at least that level of stability of the
>>     serialized form of lambda expressions.
>>
>>


-- 
- DML