From alex.buckley at oracle.com Mon Aug 5 17:00:12 2013 From: alex.buckley at oracle.com (Alex Buckley) Date: Mon, 05 Aug 2013 17:00:12 -0700 Subject: [type-annos-observers] Desugaring of anonymous classes In-Reply-To: <5159E70D.2040702@oracle.com> References: <20130331.113913.2061923468052370653.mernst@cs.washington.edu> <5159E70D.2040702@oracle.com> Message-ID: <52003C8C.2010901@oracle.com> Mike, Srikanth, Returning to this topic, there are two scenarios where I would like your comments: 1. Type annotation in 'new' expression for anonymous class Consider this code: @Retention(RetentionPolicy.RUNTIME) @Target(ElementType.TYPE_USE) @interface X {} class C { void m() { new @X Foo() {}; } } The @X annotation is stored as a target_type=CLASS_EXTENDS structure in the RuntimeVisibleTypeAnnotations attribute of Foo$1.class. That's fine. The @X annotation is not stored as a target_type=NEW structure in the RuntimeVisibleTypeAnnotations attribute of m's method_info in C.class. That's an oversight by javac, especially since the first mail in this thread said javac copied type annotations "to the class declaration and to its instantiation (new expression)". 2. Declaration annotation for anonymous class in 'new' expression Take the code above and change the annotation type's target to be TYPE: @Retention(RetentionPolicy.RUNTIME) @Target(ElementType.TYPE) @interface X {} class C { void m() { new @X Foo() {}; } } The @X annotation is stored in the RuntimeVisibleAnnotations structure of Foo$1.class. This is a neat trick, since anonymous class declarations could not be annotated in SE 7, but it is irregular because annotations on declarations traditionally appear as modifiers. If a 'new' expression is to provide a declaration context (for TYPE annotations) as well as a type context (for TYPE_USE annotations), then the spec needs to spell it out. What's more, annotations on anonymous class declarations could have implications for core reflection and other language features. It appears that "new @X @X Foo() {};" works at compile-time, but I have not tried reflecting over Foo$1.class's RuntimeVisibleAnnotations attribute. Alex On 4/1/2013 12:59 PM, Alex Buckley wrote: > On 3/31/2013 11:39 AM, Michael Ernst wrote: >> In January, I removed the text from the specification that was >> compiler-specific. This text was among the removed text: >> >> As another example, an anonymous class desugars into a class >> declaration >> and instantiation. A programmer may write both type and declaration >> annotations on an anonymous class in Java source code. In the >> classfile, >> the type annotations are copied to the class declaration and to its >> instantiation (new expression). The declaration annotations are >> copied >> only to the class declaration. >> >> Do you consider this possibly implementation-specific, and therefore it >> does not belong in the specification. Or, will every compiler follow >> this >> implementation strategy so it is worth re-instating as useful guidance? > > It would be better to start with the concept of a class instance > creation expression (CICE) since it lets the locations be more precise. > > A CICE may create an instance of an anonymous class, in which case the > declaration of the anonymous class is provided inline. This declaration > is required by JLS 13.1 to compile to a form with a binary name; you can > take this to mean a standalone class file is emitted. > > A CICE may have i) type annotations in the 'new ... {' term outside the > anonymous class declaration, and ii) type annotations and declaration > annotations inside the anonymous class declaration. It would be > appropriate to specify: > > - The (i) annotations (type only) are stored for the CICE in its > enclosing class file _and_ stored for the superclass of the anonymous > class declaration in its emitted class file. > > - The (ii) annotations (type and decl) are stored for the appropriate > type uses and declarations in the class file emitted for the anonymous > class declaration. > > Alex From alex.buckley at oracle.com Mon Aug 5 18:36:25 2013 From: alex.buckley at oracle.com (Alex Buckley) Date: Mon, 05 Aug 2013 18:36:25 -0700 Subject: [type-annos-observers] JLS and JVMS changes for JSR 308 Message-ID: <52005319.9000209@oracle.com> I have prepared a PDF containing the JLS and JVMS changes for JSR 308: http://cr.openjdk.java.net/~abuckley/308.pdf If you're expecting a simple list of grammar changes, you may want to sit down before opening the PDF. There are significant clarifications to JLS chapters 4 and 6, all aimed at pinning down what exactly is a type use. The JLS has distinguished between types (a.k.a. type uses) and type names since JLS3, but it was very subtle, and it is essential for JLS8 to take a comprehensive view of the language's syntactic contexts and the types and names allowed therein. The ClassFile changes present a recurring challenge: how much to mention Java language features in the JVMS? The Signature attribute is a precedent, so for now, I have aimed for maximum clarity. An example would be mentioning both constructors and methods when describing target_info, even though the ClassFile structure is aware only of methods. Text in the PDF is either normative or informative. Informative text is smaller and indented. The point of the PDF is that most text is ready to go into the JLS or JVMS as-is, although informative text which mentions JSR 308 is intended more as commentary for 2013 than as long-term specification; it will evolve further, or be removed. The UW PDF at: http://types.cs.washington.edu/jsr308/specification/java-annotation-design.pdf remains an excellent source of background, examples, and rationale. This kind of material should be available on the JSR 308 page at jcp.org forever, so I envisage offering both 308.pdf and java-annotation-design.pdf in the milestone reviews yet to come. Mike, it would be instructive to look at how JSR 334's specification changed between Public Review and Proposed Final Draft. Vast swathes of normative text were replaced with pointers to JLS sections. It would be appropriate to start the same "pruning" process on java-annotation-design.pdf. This will provide a double check on my extraction of normative clauses from java-annotation-design.pdf. Alex From srikanth_sankaran at in.ibm.com Fri Aug 23 02:33:37 2013 From: srikanth_sankaran at in.ibm.com (Srikanth S Adayapalam) Date: Fri, 23 Aug 2013 15:03:37 +0530 Subject: [type-annos-observers] Desugaring of anonymous classes In-Reply-To: <52003C8C.2010901@oracle.com> References: <20130331.113913.2061923468052370653.mernst@cs.washington.edu> <5159E70D.2040702@oracle.com> <52003C8C.2010901@oracle.com> Message-ID: > From: Alex Buckley > Sent by: type-annotations-spec-experts-bounces at openjdk.java.net > > Mike, Srikanth, > > Returning to this topic, there are two scenarios where I would like your > comments: Hi Alex, First of all, apologies for the delay in responding. > 1. Type annotation in 'new' expression for anonymous class > > Consider this code: > > @Retention(RetentionPolicy.RUNTIME) > @Target(ElementType.TYPE_USE) > @interface X {} > > class C { void m() { new @X Foo() {}; } } > > The @X annotation is stored as a target_type=CLASS_EXTENDS structure in > the RuntimeVisibleTypeAnnotations attribute of Foo$1.class. That's fine. I am fine with this behavior, though as there is nothing in the latest specification that calls for such a behavior, we should make sure to spell this out, if this is the behavior we want. > The @X annotation is not stored as a target_type=NEW structure in the > RuntimeVisibleTypeAnnotations attribute of m's method_info in C.class. > That's an oversight by javac, especially since the first mail in this > thread said javac copied type annotations "to the class declaration and > to its instantiation (new expression)". At the moment, eclipse compiler's behavior is the exact opposite of what you mention as javac's behavior. We emit the annotation with target_type being NEW but not target_type=CLASS_EXTENDS. > 2. Declaration annotation for anonymous class in 'new' expression > > Take the code above and change the annotation type's target to be TYPE: > > @Retention(RetentionPolicy.RUNTIME) > @Target(ElementType.TYPE) > @interface X {} > > class C { void m() { new @X Foo() {}; } } > > The @X annotation is stored in the RuntimeVisibleAnnotations structure > of Foo$1.class. This is a neat trick, since anonymous class declarations > could not be annotated in SE 7, but it is irregular because annotations > on declarations traditionally appear as modifiers. If a 'new' expression This is more irregular than neat and I would be inclined to suggest that we pull back such behavior. At the moment again, as you call out there is nothing in the specification that calls for such a behavior. Srikanth, From alex.buckley at oracle.com Fri Aug 23 12:04:56 2013 From: alex.buckley at oracle.com (Alex Buckley) Date: Fri, 23 Aug 2013 12:04:56 -0700 Subject: [type-annos-observers] Desugaring of anonymous classes In-Reply-To: References: <20130331.113913.2061923468052370653.mernst@cs.washington.edu> <5159E70D.2040702@oracle.com> <52003C8C.2010901@oracle.com> Message-ID: <5217B258.80402@oracle.com> Hi Srikanth, On 8/23/2013 2:33 AM, Srikanth S Adayapalam wrote: > > 1. Type annotation in 'new' expression for anonymous class > > > > Consider this code: > > > > @Retention(RetentionPolicy.RUNTIME) > > @Target(ElementType.TYPE_USE) > > @interface X {} > > > > class C { void m() { new @X Foo() {}; } } > > > > The @X annotation is stored as a target_type=CLASS_EXTENDS structure in > > the RuntimeVisibleTypeAnnotations attribute of Foo$1.class. That's fine. > > I am fine with this behavior, though as there is nothing in the latest > specification that calls for such a behavior, we should make sure to > spell this out, if this is the behavior we want. Foo is, of course, the direct superclass of the anonymous class (JLS 15.9.1), but the 'extends' clause is hard to see in the code above. The wording for target_type=CLASS_EXTENDS (0x10) refers to a "type in extends clause of class or interface declaration" so it's not clear that the @X annotation should be stored this way. To clarify, the wording for target_type=CLASS_EXTENDS should say: "type in extends clause of class or interface declaration (including the direct superclass of an anonymous class declaration), ..." > > The @X annotation is not stored as a target_type=NEW structure in the > > RuntimeVisibleTypeAnnotations attribute of m's method_info in C.class. > > That's an oversight by javac, especially since the first mail in this > > thread said javac copied type annotations "to the class declaration and > > to its instantiation (new expression)". > > At the moment, eclipse compiler's behavior is the exact opposite of what > you mention as javac's behavior. We emit the annotation with target_type > being NEW but not target_type=CLASS_EXTENDS. I will file a bug for javac to emit the target_type=NEW attribute. Then if ecj can emit the target_type=CLASS_EXTENDS, everything will be good. > > 2. Declaration annotation for anonymous class in 'new' expression > > > > Take the code above and change the annotation type's target to be TYPE: > > > > @Retention(RetentionPolicy.RUNTIME) > > @Target(ElementType.TYPE) > > @interface X {} > > > > class C { void m() { new @X Foo() {}; } } > > > > The @X annotation is stored in the RuntimeVisibleAnnotations structure > > of Foo$1.class. This is a neat trick, since anonymous class declarations > > could not be annotated in SE 7, but it is irregular because annotations > > on declarations traditionally appear as modifiers. If a 'new' expression > > This is more irregular than neat and I would be inclined to suggest that > we pull back such behavior. At the moment again, as you call out there is > nothing in the specification that calls for such a behavior. Thanks for this comment. I will file a bug for javac to remove the RuntimeVisibleAnnotations attribute. In the recent JLS/JVMS draft for JSR 308, the above code is illegal: a 'new' expression is a "type context", and "It is a compile-time error if an annotation of type T decorates a type in a type context (4.11), but T is not applicable to type uses." ... the annotation type X is not applicable to type uses because "The constants ... TYPE ... correspond to the declaration contexts ...". Alex From alex.buckley at oracle.com Fri Aug 23 15:49:47 2013 From: alex.buckley at oracle.com (Alex Buckley) Date: Fri, 23 Aug 2013 15:49:47 -0700 Subject: [type-annos-observers] Storing declaration annotations on a local variable In-Reply-To: References: Message-ID: <5217E70B.1010809@oracle.com> For the record, it is too late for JSR 308 to consider storing declaration annotations on local variables. Alex -------- Original Message -------- Subject: Storing declaration annotations on a local variable Date: Mon, 18 Feb 2013 01:03:26 -0800 From: Werner Dietl To: type-annotations-spec-comments at openjdk.java.net Java 1.5 declaration annotations on local variables can be written, but are not stored in the bytecode. Java 1.8 type annotations on a local variable are stored in the bytecode. Should we store declaration annotations on local variables together with type annotations on local variables? Let's assume a declaration annotation @DA, a type annotation @TA, and this code: class Test { @TA @DA Object f = null; void foo() { @TA @DA Object l = null; } } A fragment of the output of javap -v is: java.lang.Object f; descriptor: Ljava/lang/Object; flags: RuntimeInvisibleAnnotations: 0: #8() RuntimeInvisibleTypeAnnotations: 0: #10(): FIELD ... void foo(); descriptor: ()V ... RuntimeInvisibleTypeAnnotations: 0: #10(): LOCAL_VARIABLE, {start_pc=2, length=1, index=1} (Where #8 is @DA and #10 is @TA.) That is, as expected, the field has both the declaration and the type annotation, whereas the local variable only stores the type annotation. Storing declaration annotations for local variables will enable more use cases for declaration annotations and make the language more consistent. It should be very simple to store the declaration annotation together with the type annotation, giving something like: RuntimeInvisibleTypeAnnotations: 0: #8(): LOCAL_VARIABLE, {start_pc=2, length=1, index=1} 1: #10(): LOCAL_VARIABLE, {start_pc=2, length=1, index=1} The obvious disadvantage being that we would now mix type and declaration annotations. Instead, with a bit more work, we could introduce a RuntimeInvisibleLocalVariableAnnotation that is used just for this purpose. It would use the same format as for type annotations on local variables and should be straightforward to implement. The reflection API will not be extended to expose these declaration annotations on local variables. What do you think of this proposal? cu, WMD. -- http://www.google.com/profiles/wdietl From markus_keller at ch.ibm.com Mon Aug 26 08:12:04 2013 From: markus_keller at ch.ibm.com (Markus Keller) Date: Mon, 26 Aug 2013 17:12:04 +0200 Subject: [type-annos-observers] JLS and JVMS changes for JSR 308 In-Reply-To: <52005319.9000209@oracle.com> References: <52005319.9000209@oracle.com> Message-ID: * 1.4 Special Parameter Contexts The wording at the end of "8.4.1 Formal Parameters" is not precise enough: "If the receiver parameter is present, then its name must be either 'this' or the text of a qualified 'this' expression (15.8.3) [..]" The grammar for ReceiverParameter only allows "{Identifier .} this". The java-annotation-design.html clearly says that the receiver parameter is named 'this', and then it adds a special case "In a constructor in an inner type, the receiver has a name such as Outer.this". => The spec should not say "either ... or", but it should say that the name is "this", except in a constructor in an inner class, where it is "EnclosingClassName.this". => The reference to (15.8.3) is problematic for two reasons: a) in JLS7, the "Qualified this" section is actually 15.8.4 b) The grammar in JLS7 15.8 refers to ClassName, but this rule is not defined anywhere in JLS7 (already broken in JLS2 where "Qualified 'this'" was introduced). According to the grammar in ch. 18, it's a fully-qualified name. But the 'this' in the receiver parameter can only be qualified with an identifier. * 1.7 Type Annotations The end of this section defines the attachment points of annotations, but it misses to spell out details for annotations on array types. The last bullet point is: "? If the annotation's type is applicable in the declaration context represented by D and in type contexts, then the annotation is deemed to appear as both a modifier for the declaration and a decorator for the type of the declared entity. In this case, the annotation which syntactically appears in the grammar as a modifier for the declaration is also implicitly declared to annotate the type of the declared entity." This is incorrect for array types with an unqualified element type. Example: @Foo int[] f; As a type-use annotation, @Foo does *not* annotate the type of 'f' (which is 'int[]'), but it annotates only the array's element type 'int'! Furthermore, the 308.pdf doesn't define the semantics of annotations on multi-level array types. Example: String @A[][][] a; The grammar rule for ArrayType (Type {Annotation} [ ]) parses this into a tree of ArrayTypes where the annotation @A appears on the innermost ArrayType: ArrayType: String @A[][][] + ArrayType: String @A[][] + ArrayType: String @A[] + Type: String + Annotation: @A + brackets: [] + brackets: [] + brackets: [] But as http://types.cs.washington.edu/jsr308/specification/java-annotation-design.html#array-syntax explains, that grammatical deconstruction is not the intended semantics for array types. In the language model, there's e.g. no type "String @A[]", since the array brackets are intended to be read from left to right. I.e. the component type of "a"'s declared type "String @A[][][]" is "String[][]", since the @A appears on the *first* brackets pair. It's not "String @A[][]" as one would think when studying the grammar. This is an unexpected detail that should be explained by the spec, especially since this also shows up in reflective APIs like javax.lang.model.type.ArrayType#getComponentType(), which can no longer just map to AST nodes from a Java parser, but have to correctly turn a grammar-based AST inside-out, so that in the given example, @A is attached to the outermost ArrayType, and not to the innermost component type. * 2 The Java Virtual Machine Specification The section "4.7.20.2 The type_path structure" needs to explain the left-to-right semantics for multi-level array types in the type_path > path data structure. Markus P.S.: This concern is also still open: [type-annos-observers] RetentionPolicy.CLASS too broad for TYPE_USE annotations: http://mail.openjdk.java.net/pipermail/type-annotations-spec-observers/2013-July/000172.html From: Alex Buckley To: type-annotations-spec-experts Date: 2013-08-06 03:39 Subject: [type-annos-observers] JLS and JVMS changes for JSR 308 Sent by: type-annotations-spec-observers-bounces at openjdk.java.net I have prepared a PDF containing the JLS and JVMS changes for JSR 308: http://cr.openjdk.java.net/~abuckley/308.pdf If you're expecting a simple list of grammar changes, you may want to sit down before opening the PDF. There are significant clarifications to JLS chapters 4 and 6, all aimed at pinning down what exactly is a type use. The JLS has distinguished between types (a.k.a. type uses) and type names since JLS3, but it was very subtle, and it is essential for JLS8 to take a comprehensive view of the language's syntactic contexts and the types and names allowed therein. The ClassFile changes present a recurring challenge: how much to mention Java language features in the JVMS? The Signature attribute is a precedent, so for now, I have aimed for maximum clarity. An example would be mentioning both constructors and methods when describing target_info, even though the ClassFile structure is aware only of methods. Text in the PDF is either normative or informative. Informative text is smaller and indented. The point of the PDF is that most text is ready to go into the JLS or JVMS as-is, although informative text which mentions JSR 308 is intended more as commentary for 2013 than as long-term specification; it will evolve further, or be removed. The UW PDF at: http://types.cs.washington.edu/jsr308/specification/java-annotation-design.pdf remains an excellent source of background, examples, and rationale. This kind of material should be available on the JSR 308 page at jcp.org forever, so I envisage offering both 308.pdf and java-annotation-design.pdf in the milestone reviews yet to come. Mike, it would be instructive to look at how JSR 334's specification changed between Public Review and Proposed Final Draft. Vast swathes of normative text were replaced with pointers to JLS sections. It would be appropriate to start the same "pruning" process on java-annotation-design.pdf. This will provide a double check on my extraction of normative clauses from java-annotation-design.pdf. Alex From timo.kinnunen at gmail.com Sat Aug 31 06:28:10 2013 From: timo.kinnunen at gmail.com (=?utf-8?Q?Timo_Kinnunen?=) Date: Sat, 31 Aug 2013 13:28:10 +0000 Subject: [type-annos-observers] =?utf-8?q?Thoughts_on_names_and_scoping=2C?= =?utf-8?q?_arrays=2C_annotation_targets?= Message-ID: <52221650.87cd0e0a.106d.ffff878b@mx.google.com> Hi, I?ve been following Checker Framework for a while, then recently started working on debugging a type-checking plugin for it, and so now that I feel like I?ve gotten some handle on all this type annotation business, here?s feedback and impressions. Overall my impression is that the type annotations are the important driving force pushing things forward but the spec itself is only a necessary evil, a requirement to be fulfilled. That?s not a really exciting situation, but this could be improved with changes in the scoping, nested arrays and annotation targeting areas, I think. The changes to how scoping is expressed in code is the one thing I really don?t like. The relationship between fully qualified names and imported simple names, nested classes and their enclosing classes, the pseudo-sub-package structure. These form a simple, straightforward, clean system. And now type annotations come straight into the middle of all of it. The result looks really ugly and changes which parts of code are emphasized: void innerMethod(@Any Outer. at Backer Middle. at Counted Inner this, @Any Outer. at Backer Middle. at Nullable @Counted Inner other) { ... } // Quite apropos that those are mistaken for email addresses? All those @Any and @Backer annotations deemphasize that one @Nullable. As well, the .@ syntax looks like a new dereferencing operator and makes it look like the type annotation is linked to the preceding type, not the type that follows! These problems seem like collateral damage partly from nested-array annotations and partly from declaration/type-use annotation ambiguity, so nothing much can be done here. Annotating multi-level arrays has way too much influence on the whole design for how rarely it gets used. The added complexity from annotations for each level on top of the complexity already present in nested arrays doesn?t help matters. I read in the JSR308 design a section which said this: ?The prefix notation is natural, because the type is read in exactly the same order as any Java array type. As another example, to express ?non-null array of length-10 arrays of English Strings? a programmer would write? And so I decided to try that as a test, without peeking at the answer. The results are too long for this email, but essentially I ended up with the following three declarations: // To express ?non-null array of thing (length-10 arrays of English Strings)? write public final @NonNull T[] array; ?// To express ?length-10 array of thing (English Strings)? write public final @Length(10) T[] array; ?// To express ?English String? write @Language("English") String s1 = "One"; ? To combine them I would much rather like to do the following. First, annotations should bind like this: public final @NonNull T[] array; // This should? public final @NonNull array; // ?mean this? public final <@NonNull T>[] array; // ?rather than currently like this. ? Then the rest is simple substitution: public final @NonNull T[] array; + public final @Length(10) T[] array; + @Language("English") String s1; = public final @NonNull T[] array; + public final @Length(10) <@Language("English") String>[] array; = public final @NonNull <@Length(10) <@Language("English") String>[]>[] array; Which is a non-null array of length-10 arrays of English language Strings when read in left-to-right, breadth-first order. Last one is short and quick one on @Target meta-annotations. To allow the ambiguity between field declaration and type use annotations to be resolved without extra contortions, I?d like to be able to further constrain an annotation?s @Target at each site. So: public final @Critical(@Target(FIELD)) String state; The same should be possible to simulate in annotation processors already, but codifying it in the spec would be cleaner. Also, @Target should allow the same targeting options as what the JVM supports. I don?t know why these are not revealed in the API, seems like a no-brainer. Hopefully these can be helpful. With regards, Timo Kinnunen Sent from Windows Mail