Interface evolution via virtual extension methods - Extending Interfaces By Breaking Application Logic (Dangerous)
bitter_fox
bitterfoxc at gmail.com
Thu Mar 15 00:11:46 PDT 2012
2012/3/15 Elvis Ligu <elvis_ligu at hotmail.com>
> Why? IImpl in JDK8 will not compile at all, because there are two m2() one
> return int and the other return void.
> I think you didn't got my point, or probably I didn't make it clear.
> However, private override void m2() in IImpl in
> JDK8 is invalid according to what I mean. You should have an
> implementation in IImpl if you write override..., unless
> IImpl is an abstract class or an interface (that is what I mean, or
> however what I wanted to say), and ofcourse it should
> be public override void m2(). The only one that will crash here is IImpl.
>
Yes, I didn't get your point: "overrid"den method can have a implementation.
> When you extend an API with a virtual method you should provide a
> default implementation for it. So in your example if m2() is a virtual
> method it must have a default implementation.
>
Right. It is my easy mistake.
> Because of this Processor#doProcess() will not break at all, neither the
> User#use(). So there are to choices left to
> client: a) ignore I#m2() (in other words leave it as is, and accept the
> default implementation), b) provide your own
> implementation in IImpl which means THE CLIENT IS DOING ON HIS OWN, EITHER
> HE HAS TO REFACTOR HIS CODE
> TO RESOLVE CONFLICT BETWEEN int IImpl#m2() AND void I#m2() OR GO TO CHOICE
> a) AND IGNORE IT (HE STILL WILL
> BE ABLE TO USE JDK8), OR JUST DON'T USE JDK8 AT ALL AND BE HAPPY WITH
> JDK7.
>
> To avoid misconception my intention was this: if you write override ...
> method() for a virtual method then it means you
> are going to give an implementation so it is visible to your class. If you
> have an interface with a virtual method but have
> no class implementation that overrides it then the virtual method will not
> be visible at all at this class BUT IT WILL BE
> RUNNING because there is a default implementation for it. So everyone is
> happy: API developers don't break their API,
> and client code continue to run.
>
I recognized by 3) or 4) that all of calling for "overrid"den method will
be a cause of UnsupportedOperationException.
>3) When a caller call Socket#reset(), if this method is ignored in the
underlying class implementation, in our case SecureRTPSocket, then a
UnsupportedOperationException will be thrown.
>This makes sense because, the old client code is ignoring Socket#reset()
so it simply means "I don't have any implementation for this".
>4) When there exists a void SecureRTPSocket#reset().
>Again if the caller calls void Socket#reset() and the void
SecureRTPSocket#reset() exists before void Socket#reset() then an
UnsupportedOperationException is thrown.
>This makes sense because, the old client code will not have any call
Socket#reset(), instead it could have calls like SecureRTPSocket#reset()
and this will normally work because no resolution is required against
Socket interface.
So I thought any implementation has no meaning.
But is the implementation for "overrid"den method valid as you told above?
class C implements I
{
@Override
public override void m()
{
System.out.println("A");
}
public int m()
{
System.out.println("B");
return 0;
}
}
C c = new C();
c.m(); // B
I i = c;
i.m(); // A
Is this result right?
Thank you for pointing out that I am in misunderstandings,
bitter_fox
>
> ------------------------------
> Date: Thu, 15 Mar 2012 10:42:34 +0900
> Subject: Re: FW: Interface evolution via virtual extension methods -
> Extending Interfaces By Breaking Application Logic (Dangerous)
> From: bitterfoxc at gmail.com
> To: elvis_ligu at hotmail.com
> CC: lambda-dev at openjdk.java.net
>
>
> I couldn't agree with you.
> I don't think "override" resolve the problem.
>
> For instance:
>
> @API
> @Version("JDK7")
> interface I
> {
> void m1();
> }
>
> @API
> @Version("JDK7")
> class Processor
> {
> public void doProcess(I i)
> {
> i.m1();
> }
> }
>
> @Client
> @Version("JDK7")
> class IImpl implements I
> {
> public void m1()
> {
> ...;
> }
>
> public int m2()
> {
> ...;
> }
> }
>
> @Client
> @Version("JDK7")
> class User
> {
> public void use()
> {
> new Processor().doProcess(new IImpl());
> }
> }
>
> I think this is a very frequently pattern.
>
> And in JDK8, @API Processor is updated with the update of I:
>
> @API
> @Version("JDK8")
> interface I
> {
> void m1();
>
> @Version("JDK8")
> void m2();
> }
>
> @API
> @Version("JDK8")
> class Processor
> {
> public void doProcess(I i)
> {
> i.m1();
> i.m2();
> }
> }
>
> @Client
> @Version("JDK8")
> class IImpl implements I
> {
> public void m1()
> {
> ...;
> }
>
> public int m2()
> {
> ...;
> }
>
> @Version("JDK8")
> private override void m2();
> }
>
> // User is retained at @Version("JDK7")
>
> We can compile well every classes.
> However User#use crashes in JDK8 at runtime.
> So we find "override" just postpone occurring error until runtime.
>
> And all of method invocations are in fear by "override".
> Because it breaks the contract of types: a member declared in supertype is
> valid in a instance of every subtype.
> This is critical in Java that is strong typed language and a new problem.
>
> I guess "override" might cause more trouble than it solves.
>
> Regards,
> bitter_fox
>
>
> 2012/3/14 Elvis Ligu <elvis_ligu at hotmail.com>
>
>
>
>
> From: elvis_ligu at hotmail.com
> To: brian.goetz at oracle.com
> Subject: RE: Interface evolution via virtual extension methods - Extending
> Interfaces By Breaking Application Logic (Dangerous)
> Date: Tue, 13 Mar 2012 16:40:14 +0100
>
>
>
>
>
>
>
>
>
>
>
>
>
> (I just saw my email, and I apologize for the horrible formatting)
> I will try to give a very simple example in order to explain (at least I
> will give it a try) what I mean. Before starting let me tell you that I
> don't have a lot of knowledge about compilers and how things works into
> details, so I can't propose anything in detail. I am looking at the problem
> from a SE perspective and how to achieve a smooth transition from java 7 to
> java 8 avoiding the overall complexity and accidentally breaking the client
> code. By breaking client code I mean, breaking client's application even
> from not compiling at all (which is the least undesirable) or breaking
> client's application logic (the worst case is when the client's application
> compiles, run, and pass all tests but the application logic is in some way
> compromised).
> Before continue this is how do I interpret the following concepts:
> - Client code, API's client, and client in general -> the client is some
> other (let say a developer) different from API. For example, I am a client
> of Collections API in Java SE 7, and my code is the client code. You can
> assume my classes are API's client or I (as a developer) am the client.
> Generally speaking, the client is independent from the API's development
> process, and is using the API according to its public contract (the
> interface, whether they are classes or interfaces).
> - Caller -> I would like to avoid the misunderstanding between an API's
> client and a class client. So when I say caller I mean the client of a
> specific class (and or interface). I.e. in class A there is a statement
> b.print(); in this case I would rather say class A (the instance of A) is
> the caller of class B (the instance of B). - Java Interface -> explicitly
> or implicitly a Java Interface (i.e, interface Service {}) is part of the
> contract of the API. It doesn't provide any implementation (and it
> shouldn't), and the purpose of it is to define a communication point
> between the implementation provider and the caller. An other purpose is to
> define an implicit logic, i.e. Comparator has method compare(), where the
> method signature is the definition of communication rules (the caller
> must call this method using its signature), and the logic is that the
> method compare() takes two same arguments and return the order of those
> objects: 0 -> equals, positive integer -> ob!
> j1 > obj2, negative integer -> obj1 < obj2. Of course there is nothing to
> prevent a client to break the implicit logic an interface imposes, and
> this depends on implementation, however this is not an issue from the
> API's developer perspective.
> Our example - the API: interface Socket { ... // method
> added in release 1.1 public void consumePackets(final
> PacketBuffer<Packet> packets) default { final
> List<Packet> buffer = new ArrayList<Packet>();
> synchronized(packets) { while(packets.hasNext())
> buffer.add(p.next()); } final PacketSequence sequence =
> new PacketSequence(buffer.toArray(new Packet[0])); // an
> empty sequence will be returned if no previous one exists final
> PacketSequence current = getPreviousSequence().merge(sequence);
> setCurrentSequence(current); } ... } interface SecureSocket
> extends Socket { } interface RTPSocket { ... // this is
> defined in release 2.0 public void consumePackets(final
> PacketBuffer<Packet> packets) default { final
> List<Packet> buffer = new ArrayList<Packet>();
> synchronized(packets) { while(packets.hasN!
> ext()) { Packet p = p.next();
> buffer.add(p); // send each packet to handler to avoid
> delay transmitPacketsToRTPHandlers(new Packet[]{p});
> } final PacketSequence sequence = new
> PacketSequence(buffer.toArray(new Packet[0]));
> // an empty sequence will be returned if no previous one exists
> final PacketSequence current = getPreviousSequence().merge(sequence);
> setCurrentSequence(current); } ... } class
> DefaultSocket implements Socket{} class DefaultSecureSocket extends
> DefaultSocket implements SecureSocket{} class DefaultRTPSocket extends
> DefaultSocket implemenets RTPSocket{}
> Here the API designers, have designed an API which provide socket
> communication to its clients. In release 1.0 the Socket interface didn't
> have a consumePackets() method. In release 1.1 API designers would like to
> extend Socket interface so they introduced consumePackets() and provided a
> default implementation because they don't want to break the clients' code.
> In release 2.0 they decided to change the default implementation of
> consumePackets() because they observed their DefaultRTPSocket was having
> delays when this method was called. They decided to deliver each packet to
> the handlers while consuming the packets and leave the rest as is.
> Our example - API's client
> // defined after release 1.1 of API class SecureRTPSocket extends
> DefaultSecureSocket implements RTPSocket { ... public void
> setCurrentSequence(PackeSequence sequence) {
> doubleBufferSequence(sequence);
> transmitPacketsToRTPHandlers(getPacketsArray()); }
> // this is not override, this is defined here public int reset()
> { ... }; }
> Here the API's client, using the version 1.1 decided to construct a new
> secure rtp socket. He used the default implementation of consumePackets()
> because it did meet his requirements. Because he already observed the delay
> produced by consumePackets() he decided to use a double buffer in order to
> avoid it. His application was working perfectly until he decided to replace
> the old API v1.1 with the new API v2.0. After replacing the API its
> application started to have delays! In this case the delay comes from the
> fact that API designers override the default implementation of
> consumePackets(), and the handler is getting duplicate packets!!! Of course
> the handler is smart enough to reject duplicate packets. But the problem is
> still there, so the client is going to invest it. The client is lookingat
> DefaultSecureSocket and at Socket#consumePackets() but he can't find the
> problem. Why he is not looking at RTPSocket? Well for 15 years now Java
> developers don't expect the interface to!
> impact their implementation without breaking the contract (look at the
> definition above). I know such a powerful mechanism in Java would by very
> great and using it wisely would provide so much flexibility to an API
> developer. On the other hand it increases the possibility of writing
> fragile code. Of course the above example is such a bad one and can be
> easily avoided, but again virtual methods are increasing risks. Think of
> C++ and its power, think of multiply inheritance, one can say that it is
> powerful, however (in my opinion) it complicates things and can lead at
> more fragile code. As you stated this is an old problem, because this
> problem is still present with abstract classes for example, but as I
> previously stated a Java developer has never looked at Interfaces as they
> are abstract classes. Should they start doing it now?
> As I can see you are working at Oracle, and I understand that when it
> comes to make a change to existing Java API it is a very critical decision,
> to be or not to be. One of the big problems you should face with is
> backward compatibility. I was reading some email from a guy here at this
> mail list where he was concerned about Collections API and the changes that
> can possibly be made at Java 8. He had extended a Collections interface and
> added some methods, let say foreach(). In this case if you add a method
> foreach() it would possibly break his API and this is not what you want. In
> the above example if the client had a method int reset() in SecureRTPSocket
> the API designers will not be able to add a method void reset() in Socket
> interface, at least not without breaking the client's code. The problem
> here is that API developers don't know how API clients are using their API
> so they can not make any assumption. Actually the virtual methods gives you
> the (virtual reality) sense o!
> f thinking that by adding a method to an interface you, will not break
> the API, but as you we can easily see this is not the case.
> * Here I am not criticizing the virtual methods, I like the idea very
> much. I am rising some of the facts we need to consider when dealing with
> them.
> However, I think there is an interesting solution we can avoid such
> problems, especially API breaks (from the perspective of API designer).
> Looking at the above example, let consider the API designers would like to
> add a method void reset() at Socket interface. In this case we could have a
> client's code break here, because there is a int reset() method. To avoid
> this, we can have an other keyword: override.In order the SecureRTPSocket
> to see the new virtual method he has to write something like this:
> private override void reset(){...}
> If the client doesn't write something like this than this method is not
> visible to client's class SecureRTPSocket. Some one would say this is not
> possible because what if some one makes a call like this:
> Socket socket = getSocket(); socket.reset();
> In such a case if getSocket() has a return type Socket and the
> implementation of getSocket() returns a SecureRTPSocketthere are two
> possibilities: 1) method reset() is already present in SecureRTPSocket, 2)
> method reset() is not present. Here how it can possibly works:
> 1) In version 1.1 the client didn't know any Socket#reset() method so
> there is not a statement in his code that will call Socket#reset().If the
> Socket#reset() method is ignored in v2.0 his code will continue to be
> compatible even in case there is a int SecureRTPSocket#reset().
> 2) There is a void SecureRTPSocket#reset() method. Even in this case the
> Socket#reset() method will be ignored if called as above. At least in
> client's code will not be any Socket#reset() call, because his code was
> written based on v1.1.
> 3) When a caller call Socket#reset(), if this method is ignored in the
> underlying class implementation, in our case SecureRTPSocket, then a
> UnsupportedOperationException will be thrown. This makes sense because, the
> old client code is ignoring Socket#reset() so it simply means "I don't have
> any implementation for this".
> 4) When there exists a void SecureRTPSocket#reset(). Again if the caller
> calls void Socket#reset() and the void SecureRTPSocket#reset() existsbefore
> void Socket#reset() then an UnsupportedOperationException is thrown. This
> makes sense because, the old client code will not have any call
> Socket#reset(), instead it could have calls like SecureRTPSocket#reset()
> and this will normally work because no resolution is required against
> Socket interface.
> 5) When there exists a override void DefaultSocket#reset(). Well in this
> case the void Socket#reset() will be visible to DefaultSocket, and it will
> be visible to DefaultSecureSocket and DefaultRTPSocket, and so it will be
> visible to SecureRTPSocket, because SecureRTPSocket extends
> DefaultSecureSocket. If there exist void SecureRTPSocket#reset() then it
> just overrides void DefaultSecureSocket#reset() so there is not a problem
> in this case. If there exist a int SecureRTPSocket#reset() than client's
> code will break (will not compile at all). This makes sense because when a
> client extends API's classes it means he knows what he is doing and he
> understand the internals of those classes, so he is aware of any change. In
> our example a good developer would simply uses encapsulation, and have
> something like this: class SecureRTSocket implements SecureSocket,
> RTSocket, and uses a private DefaultSecureSocket to delegate some of its
> methods and implements the rest. However, if this is n!
> ot a good approach an other alternative would be to ignore the virtual
> methods just like in the above cases. The last approach introduces an extra
> effort because it requires from API developers to write override void
> reset() to each implementation class (Default...Socket) but at least this
> ensures 100% backward compatibility. Think of void Collection#foreach, it
> will require for each Collection's subtype to write override void
> foreach(). Is this affordable? Well, the virtual methods are used to
> extends an API so I think this is an effort that it worth.
> 6) What does override void reset() implementation body contains? The case
> is: a) it can completely override the super's default implementation, b) it
> can write something like this SuperType.reset() in order to accept super's
> default implementation.
> 7) Where the client has already extended Socket interface, let say he has
> an interface like this ISecureRTPSocket and SecureRTPSocket class
> implements this interface. In this case void Socket#reset() is a virtual
> method so it is visible to ISecureRTPSocket as a virtual one. But it is not
> visible to SecureRTPSocket because the doesn't exists any override void
> SecureRTPSocket#reset(). Here we don't have any client's code compatibility
> issues, it works just like the above cases.
> 8) Where there doesn't exists any override void SecureRTPSocket#reset()
> (look at the example above). In this case if the first approach will be
> implemented then if exists override void DefaultSocket#reset() the reset()
> method will be visible to all other subclasses just like any other method.
> If the second approach will be implemented then, if there doesn't exist any
> override void DefaultSocket#reset() then it will not be visible to
> DefaultSecureSocket so a override void DefaultSecureSocket#reset() will be
> a compilation error (not in our case because DefaultSecureSocket implements
> SecureSocket, reset() is visible to SecureSocket and so override void
> DefaultSecureSocket#reset() can be overrided). The rule is simple: if we
> have interface A, B and C extends A, then virtual method A#m() will be
> visible to B and C. If classes Bimp implements B, Dimp extends Bimp, if
> there exists override Bimp#m() then could exist override Dimp#m() too, if
> not then Dimp must explicitly implements !
> A, B or C so we can have override Dimp#m().
> 9) The compiler and runtime will be crazy with all this complexity.
> Actually I can't give any reliable response to this. From a higher level I
> think the compiler and runtime can treat virtual methods just like any
> other method but with some small differences. In case there is a Interface
> to Interface resolution they can easily be treated just like other methods,
> after all we can keep backward compatibility because an interface is not
> required to have an implementation so the virtual method in my interfaces
> will not break other interfaces. In case of classes the compiler and or
> runtime can make the distinct between virtual methods and other regular
> methods by keeping (let say) an extra bit for a method. So when this bit is
> 0 it is a regular method and the resolution is the same as before in Java
> 7, when this bit is 1 it is a virtual method so the resolution can follow
> the rules I stated above. Especially if the second approach will be
> implemented (see 5) above) the compiler c!
> an easily keep track of virtual methods because the keyword override will
> be present in any class that implements the virtual method so it can deal
> with their (strange) resolutions and conflicts.
> Conclusions:
> I think the approach of keyword override and the resolution of virtual
> methods in this way doesn't break any OOP principles (well I am just an
> undergraduate student that is going to be graduated soon, so I cant say
> this for sure :-)). Indeed it imposes good OOP because it requires the
> attention of programmer. Think of this scenario: I am about to work at a
> company and my boss just gave me a job description (the interface, which is
> the contract). I take my responsibilities and try to do my best to do my
> job according to its description. My boss has a new job description for me
> and he required to meet me to announce that. In the meanwhile some guy from
> staff who already knows that my by boss is having a new job description for
> me comes to me and requires me to do some job that is in the new job
> description not in the one I have. What should I do in this case? Should I
> try to do what he says, at the risk of doing something that I am not
> prepare to do, or should I say to him Uns!
> upportedJobException? When I get my new job description from my boss I
> sit down and learn how to do it and write to my notes override
> JobDesc#newJob1(), override JobDesc#newJob2() and so on. Next time the guy
> from staff comes to me I would be prepared and happily do what he requires
> by taking my responsibilities.
> Thank you for hearing me. I know there are brilliant minds out there that
> could make all of my ideas useless, unfortunately I am not an experienced
> Software Engineer (well actually not even a new SE :-)) so I could not
> think of all the tricks. At least I tried to squeeze my little mind and
> express my thoughts (damn English I really want to master you :-).
> I am looking forward to hearing from you.
>
> > Subject: Re: Interface evolution via virtual extension methods -
> Extending Interfaces By Breaking Application Logic (Dangerous)
> > From: brian.goetz at oracle.com
> > Date: Sun, 11 Mar 2012 21:51:35 -0700
> > CC: lambda-dev at openjdk.java.net
> > To: elvis_ligu at hotmail.com
> >
> > Thanks for your comments. Responses inline:
> >
> > > 1 - There is a risk of breaking client code: consider a developer who
> has already implemented a comparator like this: abstract class
> MyComparator implements Comparator { protected int order = 1; public void
> reverseOrder() { return -1*order;} }
> > > and uses this somewhere in his code like this:
> MyComparator c = new MyComparator() { public int compare(Objct o1, Object
> o2) { return order* (o1.toString().compareTo(o2.toString())); }} ....
> c.reverseOrder(); Arrays.sort(list, c); If oracle developers
> decide to add such a method in Comparator interface then they can not
> ensure that client code will not break; two methods with the same
> signature but with a different returned type can not compile (at least not
> in java 7).
> >
> > Yes, this is a risk. Note that this is not new with extension methods;
> this risk exists for adding new methods to existing classes (abstract or
> concrete) too.
> >
> > The alternative is: never add any new methods to libraries. We find
> this alternative unappealing!
> >
> > > 2 - There is a risk of breaking client logic: according to Brian's
> paper, section 3. Method Dispatch, we can have a diamond-shaped hierarchy
> as:
> > > interface A { void m() default X.a; } interface B extends A { }
> interface C extends A { } class D implements B, C { }
> > > When the compiler sees d.m() and if there is not an implementation of
> m() in class D then it will call the default method implementation A.m().
> >
> > The compiler does *not* short-circuit calls to D.m() to A.m(). The
> compiler emits an invokespecial with D.m(); the runtime will perform the
> resolution, and if the situation is still like this by the time the classes
> are actually loaded, then the VM will decide to dispatch to A.m(). (This
> is no different than today; the compiler's job is simply to sanity-check
> linkage and "fail fast" if the compiler can see the call would not succeed
> if made at runtime with the classes as they are at compile time, but the
> runtime builds vtables based on the classes it sees at runtime.)
> >
> > > The client of this API can easily "make the mistake" of relying on
> default implementation of A.m().
> >
> > Again, this is nothing new. Let's say class C has a method q(), and D
> extends C but does not override q(). Clients of D could easily "make the
> mistake" of assuming calls to D.q() are going to end up at C.q(), but that
> is simply a mistake.
> >
> > > For some reason in the future the developerof the API would like to
> add a default implementation of m() in B. The problem in this case is
> bigger than in case 1;
> > > a) if the client of API have already extended interface A with B1 and
> provided a default implementation for m, and has a "class E implements B,
> B1" the client code will break.
> >
> > At this point E will not compile, due to competing implementations in B
> and B1.
> >
> > > b) the default m() in B will probably break invariants of a "class
> MyClass implements B, C" and cause no compile error.
> >
> > Why? If B.m() overrides A.m(), we expect B.m() to implement the
> contract of A.m(). Note that interfaces have no state (barring heroic
> actions), so the protection of state-based invariants still falls entirely
> to C and its subclasses. The implementation of m() in A or B can only call
> other interface methods, and if any of those affect state in C, eventually
> we'll be calling code in C to actually touch the state.
> >
> > > 1 - How to keep backward compatibility? Well at least in the
> transition phase from java 7 to java 8 nothing can ensure the java
> developers that adding a virtual method inCollection interface will be
> compatible with the old java. So my suggestion is another keyword, let say
> override. Let say we add reverseOrder() in interface Comparator.In this
> case the java compiler will make the reverseOrder() visible to Comparator's
> subtypes only and only if the subtype explicitly define: override
> reverseOrder().
> >
> > I believe this is a request for "opting in" to extension methods, rather
> than having them actually inherited? I invite you to propose exactly how
> this might work in more detail.
> >
> > > So incase 1 MyComparator (the client code) is not affected;
> reverseOrder() is not visible in MyComparator. 2 - How to avoid making API
> fragile by default (case 2)? My suggestion is the same as the above: the
> keyword override. By writing override reverseOrder() the API's clientis
> required to explicitly define which default implementation he is relying
> on, specifying SuperType.super.m() or make a "default none" in case of an
> interface. If the clientdoesn't specify override, there i!
> >
> > You say "client" when I think you mean "subclass". Are you really
> suggesting the opt-in should be at the *client* site, or are you speaking
> only at the inheritance site?
> >
>
>
>
>
>
>
More information about the lambda-dev
mailing list