Stability of lambda serialization
David M. Lloyd
david.lloyd at redhat.com
Tue Jul 30 06:15:57 PDT 2013
I realize I was ambiguous:
On 07/29/2013 07:40 PM, David M. Lloyd wrote:
> As before, I continue to strongly disagree with this so-called "de
> facto" constraint. In reality, we see real users who rely heavily on
> the *real* constraints which are outlined in the specification of
> serialization. I think that real-world users are going to encounter
> bizarre issues at best, and security vulnerabilities at worst, due to
> the behavior of capture.
...due to the behavior of capture with respect to serialization.
> As I always do when this particular truism is brought up, I will counter
> your "the perfect is the enemy of the good" with "the good is the enemy
> of the correct". A clever saying is never enough to justify incorrect
> behavior. I think it's perfectly reasonable to capture "this", at most,
> but beyond that I think there is no benefit to having capture, at all,
> in the face of its instability.
...having capture in the context of serialization...
> I think the "equivalent stability" argument is also a bit fallacious.
> It's saying "anonymous classes are bad, so that's a free pass to do this
> similarly badly as well, in whatever way we want". To be "as good as
> anonymous classes", we'd have to have capture which is stable by name at
> least.
...capture in the context of serialization...
> But I personally I think this criteria is arbitrary and unnecessary in
> any case. I think that we can easily guarantee stability of both
> captured variables and of method resolution by only allowing
> serialization of named method references and forbidding capture (other
> than "this").
...forbidding capture in serializable lambdas...
> On 07/27/2013 12:18 PM, Brian Goetz wrote:
>> Yes, when this came up the first several times, the consensus of the EG
>> was that we would build on the de-facto, practical constraint that
>> people have learned to live with for years when dealing with
>> serialization in the presence of inner classes: make sure the same
>> classfile bits are on both sides of the pipe.
>>
>> Inner classes have a naming instability, where you have a stable method
>> name but an unstable class name. Lambdas have the opposite, but
>> effectively equivalent instability: a stable class name but an unstable
>> method name. People have learned to use serialization with inner
>> classes by the technique of keeping the classfile bits the same on both
>> sides of the wire. The exact same technique enables serialization of
>> lambdas.
>>
>> In particular, the EG rejected the extremes of "make it perfect" and
>> "don't allow it at all" as being "the perfect is the enemy of the good."
>> Inner classes have been exactly this hostile to serialization for 15
>> years, but people who want to use serialization have learned coping
>> techniques to do so. The compromises in the middle are squishy but
>> better than either extreme.
>>
>> Of course, if we can make it better without exposing additional
>> complexity, so much the better. For example, we did a few rounds of
>> this by encoding the hash of the method name and signature so that
>> simply reordering the methods in a class did not trigger such
>> instabilities. But the goal was never perfect stability; it was just
>> "at least as good as inner classes." I believe we've reached that goal.
>>
>> Do you have a concrete proposal for how we would improve lambda
>> serialization to be robust in the face of changes in order of capture?
>> On the surface it looks entirely possible.
>>
>> On 7/27/2013 12:52 PM, Sam Pullara wrote:
>>> I think the requirement should be that you have the same classes on each
>>> side of the wire.
>>>
>>> Sam
>>>
>>>
>>> On Tue, Jul 23, 2013 at 10:35 AM, Scott Stark <sstark at redhat.com
>>> <mailto:sstark at redhat.com>> wrote:
>>>
>>> Red Hat has a concern regarding how fragile the default
>>> serialization behavior of lambda expressions is in the current
>>> reference implementation, currently:
>>> ironmaiden:OpenJDK starksm$
>>>
>>> /Library/Java/JavaVirtualMachines/jdk1.8.0.jdk/Contents/Home/bin/java
>>> -version
>>> java version "1.8.0-ea"
>>> Java(TM) SE Runtime Environment (build 1.8.0-ea-b98)
>>> Java HotSpot(TM) 64-Bit Server VM (build 25.0-b40, mixed mode)
>>>
>>> The problem is that the serialized form of a lambda expression
>>> depends on the order in which captured arguments are declared. The
>>> attached simple example demonstrates how easy it is for a trivial
>>> reordering of the lambda code block to result in an inability to
>>> deserialize a previously saved expression.
>>>
>>> To produce this exception:
>>> 1. Run the serialization.AuthenticationContext.testWriteLambda
>>> method with the lambda expression written as:
>>> Authenticator a = (Authenticator & Serializable) (String
>>> principal, char[] pass) -> {
>>> // Run with p declared first when writing out the
>>> /tmp/testWriteLambda.bin, then switch
>>> // to declare u first when running testReadLambda
>>> String p = "-> Password " + password + " <-";
>>> String u = "-> User " + user + " <-";
>>> return u + " " + p;
>>> };
>>> 2. Change the lambda expression to:
>>> Authenticator a = (Authenticator & Serializable) (String
>>> principal, char[] pass) -> {
>>> // Run with p declared first when writing out the
>>> /tmp/testWriteLambda.bin, then switch
>>> // to declare u first when running testReadLambda
>>> String u = "-> User " + user + " <-";
>>> String p = "-> Password " + password + " <-";
>>> return u + " " + p;
>>> };
>>>
>>> Recompile and run serialization.AuthenticationContext.testReadLambda
>>> to produce:
>>>
>>> java.io.IOException: unexpected exception type
>>> at
>>>
>>> java.io.ObjectStreamClass.throwMiscException(ObjectStreamClass.java:1538)
>>>
>>> at
>>>
>>> java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1110)
>>> at
>>>
>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1807)
>>>
>>> at
>>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
>>> at
>>> java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
>>> at
>>>
>>> serialization.AuthenticationContext.testReadLambda(AuthenticationContext.java:34)
>>>
>>>
>>> ...
>>> Caused by: java.lang.reflect.InvocationTargetException
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>> Method)
>>> at
>>>
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>
>>>
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>
>>>
>>> at
>>>
>>> java.lang.invoke.SerializedLambda.readResolve(SerializedLambda.java:222)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>> Method)
>>> at
>>>
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>
>>>
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>
>>>
>>> at
>>>
>>> java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1104)
>>> ... 30 more
>>> Caused by: java.lang.IllegalArgumentException: Invalid lambda
>>> deserialization
>>> at
>>>
>>> serialization.AuthenticationContext.$deserializeLambda$(AuthenticationContext.java:1)
>>>
>>>
>>> ... 40 more
>>>
>>> One does not see the same level of sensitivity to the ordering of
>>> the serialization fields in a POJO as demonstrated by the
>>> serialization.AuthenticationContext.testWritePOJO/testReadPOJO cases
>>> where one can reorder the TestPOJO.{user,password} fields without
>>> having serialization fail.
>>>
>>> We would like to see at least that level of stability of the
>>> serialized form of lambda expressions.
>>>
>>>
>
>
--
- DML
More information about the lambda-spec-experts
mailing list