From manoj.palat at in.ibm.com Fri Apr 5 05:51:44 2019 From: manoj.palat at in.ibm.com (Manoj Palat) Date: Fri, 5 Apr 2019 11:21:44 +0530 Subject: Nestmates with invokespecial Message-ID: Hi Alex, Dan, all, Consider the following code: class X { public class Y{ private void foo() { System.out.println("Hello"); } } public class Z1 extends Y { public class Z2 { void bar() { Z1.super.foo(); // This call } } } public static void main(String[] args) { new X().new Z1().new Z2().bar(); } } Consider the call Z1.super.foo(); At ecj, for 11+, currently we do not create synthetic accessor methods for private nestmate members; instead, emit ?invokespecial? since both Y and Z2 are nestmates ? the relevant portion of the byte code is shown below: void bar(); descriptor: ()V flags: (0x0000) Code: stack=1, locals=1, args_size=1 0: aload_0 1: getfield #10 // Field this$1:LX$Z1; 4: invokespecial #20 // Method X$Y.foo:()V 7: return Run with any of the latest Java versions (I used the latest Java 12 as well); the following error is thrown: Exception in thread "main" java.lang.VerifyError: Bad invokespecial instruction: current class isn't assignable to reference class. Exception Details: Location: X$Z1$Z2.bar()V @4: invokespecial Reason: Error exists in the bytecode Bytecode: 0000000: 2ab4 000a b700 14b1 at X.main(X.java:14) Is this a problem in verification of invokespecial, where the nestmates are not considered? Or are we supposed to go via synthetic emulation for qualified super class private methods even if they are nestmates? Note: at JVMS 11 - 4.9.2, under invokespecial, I don?t see the nestmates referred to, although the corresponding bug https://bugs.openjdk.java.net/browse/JDK-8177020 mentions that the ?proposed changes? include the addition of ?With nestmates it may also name a method of a nestmate's class. ? to section 4.9.2. Any reason(s) for the omission of this condition? Regards, Manoj. Eclipse Java Dev, IBM. From alex.buckley at oracle.com Fri Apr 5 19:16:00 2019 From: alex.buckley at oracle.com (Alex Buckley) Date: Fri, 05 Apr 2019 12:16:00 -0700 Subject: Nestmates with invokespecial In-Reply-To: References: Message-ID: <5CA7A970.1010804@oracle.com> // Dropping jls-jvms-spec-comments for the time being. The code, better formatted: class X { public class Y { private void foo() { System.out.println("Hello"); } } public class Z1 extends Y { public class Z2 { void bar() { Z1.super.foo(); // This call } } } public static void main(String[] args) { new X().new Z1().new Z2().bar(); } } On 4/4/2019 10:51 PM, Manoj Palat wrote: > Consider the call Z1.super.foo(); At ecj, for 11+, currently we do not > create synthetic accessor methods for private nestmate members; instead, > emit ?invokespecial? since both Y and Z2 are nestmates ? the relevant > portion of the byte code is shown below: > > void bar(); > descriptor: ()V > flags: (0x0000) > Code: > stack=1, locals=1, args_size=1 > 0: aload_0 > 1: getfield #10 // Field this$1:LX$Z1; > 4: invokespecial #20 // Method X$Y.foo:()V > 7: return To be clear about what you mean by "nestmates": Z2.class has a NestHost attribute indicating X, and X.class has a NestMembers attribute including Z2. > Run with any of the latest Java versions (I used the latest Java 12 as > well); the following error is thrown: > > Exception in thread "main" java.lang.VerifyError: Bad invokespecial > instruction: current class isn't assignable to reference class. > Exception Details: > Location: > X$Z1$Z2.bar()V @4: invokespecial > Reason: > Error exists in the bytecode > Bytecode: > 0000000: 2ab4 000a b700 14b1 > > at X.main(_X.java:14_) > > Is this a problem in verification of invokespecial, where the nestmates > are not considered? Or are we supposed to go via synthetic emulation for > qualified super class private methods even if they are nestmates? > > Note: at JVMS 11 - 4.9.2, under invokespecial, I don?t see the nestmates > referred to, although the corresponding bug > _https://bugs.openjdk.java.net/browse/JDK-8177020_mentions that the > ?proposed changes? include the addition of ?With nestmates it may also > name a method of a nestmate's class. ? to section 4.9.2.**Any reason(s) > for the omission of this condition? In JDK-8177020, there are notable changes between the April 2017 spec (private-access.html) and the May 2018 spec (nestmates.html). In line with the bug's Description, the 2017 spec had a lot to say about invokespecial. However, the 2018 spec took a different tack for privat methods. In nestmates.html, see the commentary on a change in 3.7: "Because private methods may now be invoked from a nestmate class, it is no longer recommended to compile their invocation to invokespecial. (invokespecial may only be used for methods declared in the current class or a superclass.) A standard usage of invokevirtual or invokeinterface works just fine instead, with no special discussion necessary." Accordingly, JVMS 11 didn't change invokespecial's verification; the relevant high-level rule in 4.9.2 is still "If an invokespecial instruction names a method which is not an instance initialization method, then the target reference on the operand stack must be a class instance whose type is assignment compatible with the current class." Frankly, JEP 181 presented a lot of low-level rule changes to invoke{special,virtual,interface} but didn't do a great job of presenting a single high-level description of intent. For example, nestmates.html flags the major change of "Modifying the behavior of invokeinterface to support invocation of private methods.", but that simply raises a hundred questions, which could be addressed by (say) a matrix of bytecodes and kinds-of-method. I hope Dan can share some thoughts, on this list, of the full invoke* story intended by JEP 181. Alex From alex.buckley at oracle.com Fri Apr 5 19:19:08 2019 From: alex.buckley at oracle.com (Alex Buckley) Date: Fri, 05 Apr 2019 12:19:08 -0700 Subject: Nestmates with invokespecial In-Reply-To: <5CA7A970.1010804@oracle.com> References: <5CA7A970.1010804@oracle.com> Message-ID: <5CA7AA2C.8070508@oracle.com> On 4/5/2019 12:16 PM, Alex Buckley wrote: > // Dropping jls-jvms-spec-comments for the time being. Ignore this part. From daniel.smith at oracle.com Mon Apr 8 23:44:47 2019 From: daniel.smith at oracle.com (Dan Smith) Date: Mon, 8 Apr 2019 17:44:47 -0600 Subject: Nestmates with invokespecial In-Reply-To: <5CA7A970.1010804@oracle.com> References: <5CA7A970.1010804@oracle.com> Message-ID: I assume that, in general, you've switched to 'invokevirtual' for private methods, per the discussion in the JEP and the spec document. What's tricky about this example is that you don't want a virtual invocation?you need the 'super' invocation mode. So 'invokevirtual' is no good. And what you need to note is that JEP 181 didn't give any new capabilities to invokespecial (aside from indirect benefits from new accessibility rules)?it can only invoke i) constructors and ii) methods of the caller/its supers. Doesn't matter whether the referenced method is accessible or not. So, when it comes to 'super' invocations of the methods of an outer class's supertypes, you'll need to continue generating bridges (whether the outer's super is in the same nest or not). --- To the request to unpack some of the changes: > On Apr 5, 2019, at 1:16 PM, Alex Buckley wrote: > > Frankly, JEP 181 presented a lot of low-level rule changes to invoke{special,virtual,interface} but didn't do a great job of presenting a single high-level description of intent. For example, nestmates.html flags the major change of "Modifying the behavior of invokeinterface to support invocation of private methods.", but that simply raises a hundred questions, which could be addressed by (say) a matrix of bytecodes and kinds-of-method. I hope Dan can share some thoughts, on this list, of the full invoke* story intended by JEP 181. (Meta-note: yes, the artifacts we've produced are somewhat hard to follow. We should do a better job of laying out a high-level view of the changes in the JEP and/or the spec document header.) I can try to offer some clarity. The header of nestmates.html describes two changes, and that is a good place to start: 1) A private member in another class may now be accessible, by virtue of the NestHost and NestMembers attributes of the caller/callee. This means that some bytecode that would have previously resulted in an IAE due to an inaccessible private member now succeeds. The description is not precise about referenced vs. declaring classes; the key definitions are in 5.4.4. The trick about the invokespecial instruction in question: > invokespecial #20 // Method X$Y.foo:()V is that it was formerly illegal for *two* reasons: - The referenced method is private and not accessible to Z2 - The referenced method is not declared in the caller or its supers. Now it's illegal for just one reason. But still illegal. (Why didn't we "fix" that? Because fixing it properly is hard and complex, and we didn't feel doing so was justified.) These accessibility changes affect all instructions that rely on member references *except for invokeinterface* (discussed next), including invokevirtual, invokestatic, and invokespecial. 2) The invokeinterface instruction may be used to invoke private methods. A former rule explicitly required that the method referenced by invokeinterface be public. We relaxed that, allowing it to be private as well, thus able to take advantage of the changes in (1) (if the private method turns out to be accessible). Those are really the only meaningful changes (aside from some cornery changes called out in the discussion boxes). In particular, while there are refactorings to improve the presentation of the invoke* instructions, there are no meaningful changes to them, except as discussed above.