From iris.clark at oracle.com Mon Mar 4 18:38:11 2019 From: iris.clark at oracle.com (Iris Clark) Date: Mon, 4 Mar 2019 10:38:11 -0800 (PST) Subject: JSR 388 (Java SE 13) Specification -- DRAFT 10 Message-ID: <37ee898f-4b31-4d06-8caa-6c7ce60a797d@default> Hi. An updated draft of the Specification is available here: https://cr.openjdk.java.net/~iris/se/13/spec/draft/java-se-13-draft-spec-10/ This draft includes the following changes: - Annexes 1 and 2: Updates based on build 10 Once JEPs are Targeted for JDK 13, I'll add the "Feature summary" and "Feature details" sections. Similarly, if any APIs or non-API features are removed, I'll add the "APIs removed" or "Non-API features removed" sections as appropriate. This JSR uses JCP version 2.11 as defined in JSR 387. The next JCP milestone is PR, scheduled for Summer 2019. Until then, draft releases of the Specification and Reference Implementation will be provided on a regular basis. Links to the JEP/CSR Dashboards, latest Specification, latest JavaDoc API and diffs, and latest RI binaries may be found on this page: https://openjdk.java.net/projects/jdk/13/spec/ Thanks, iris From iris.clark at oracle.com Mon Mar 11 06:28:14 2019 From: iris.clark at oracle.com (Iris Clark) Date: Sun, 10 Mar 2019 23:28:14 -0700 (PDT) Subject: JSR 388 (Java SE 13) Specification -- DRAFT 11 Message-ID: Hi. An updated draft of the Specification is available here: https://cr.openjdk.java.net/~iris/se/13/spec/draft/java-se-13-draft-spec-11/ This draft includes the following changes: - Annexes 1 and 2: Updates based on build 11 Once JEPs are Targeted for JDK 13, I'll add the "Feature summary" and "Feature details" sections. Similarly, if any APIs or non-API features are removed, I'll add the "APIs removed" or "Non-API features removed" sections as appropriate. This JSR uses JCP version 2.11 as defined in JSR 387. The next JCP milestone is PR, scheduled for Summer 2019. Until then, draft releases of the Specification and Reference Implementation will be provided on a regular basis. Links to the JEP/CSR Dashboards, latest Specification, latest JavaDoc API and diffs, and latest RI binaries may be found on this page: https://openjdk.java.net/projects/jdk/13/spec/ Thanks, iris From volker.simonis at gmail.com Mon Mar 18 15:59:55 2019 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 18 Mar 2019 16:59:55 +0100 Subject: Certification of Preview Features Message-ID: Hi, we've recently started the TCK certification for Java SE 12. As we all know, Java SE 12 is the first release with "Preview Features". As we've previously discussed and as described in the "JEP 12: Preview Language and VM Features" [1], preview features are fully specified and fully implemented features of the Java SE platform which have to be certified by the corresponding TCK. With the latest Java SE 12 TCK, this testing looks as follows (see attached image): - one has to choose whether to run the test on the preview features or not - the documentation mentions that one "may need to test the product in both modes" First of all, "may need" is a much to weak requirement. According to JEP 12 and our discussions around the topic I'm pretty sure that a Java SE 12 compliant implementation will "HAVE TO" run the tests in both modes. Second, the TCK doesn't mention which tests will have to be run in "both modes". Running all the runtime and langtools tests (they have the same "preview feature" switch) is a real big overhead. It actually doubles the time for a correct certification (which already is quite substantial). It also interferes with other test switches like "verify/noverify" which already require to run a part of the tests in two different configurations. Together with the preview features, we will now theoretically get four configurations which will have to be tested. The only practical solution to this problem would be to change the TCK such that it automatically runs all the required tests affected by the preview features in both configurations. Running all available tests in both configurations makes no sense and is a complete waste of time (e.g. why should one have to run all the interactive AWT/Swing or printing tests two times, with and without preview features enabled if they are not affected at all by these features?). Finally, I do realize now that my proposal to use preview features at least within the reference implementation itself, in order to gather some experience with the new features, doesn't play very well with certification. Also, if the JDK itself would make use of such features, it would only run with "--enable-preview" which is probably not what we want (unless we find a way to enable preview features transparently for the JDK classes only). What do you think? Regards, Volker [1] http://openjdk.java.net/jeps/12 From brian.goetz at oracle.com Mon Mar 18 17:01:26 2019 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 18 Mar 2019 13:01:26 -0400 Subject: Certification of Preview Features In-Reply-To: References: Message-ID: <0FCF5BC1-DA97-46AA-B665-17DDB8F20D1E@oracle.com> > With the latest Java SE 12 TCK, this testing looks as follows (see > attached image): Image was stripped. > First of all, "may need" is a much to weak requirement. According to > JEP 12 and our discussions around the topic I'm pretty sure that a > Java SE 12 compliant implementation will "HAVE TO" run the tests in > both modes. Yes, this seems more in the spirit of preview features. More precisely, some tests _must_ run with preview features enabled (in order to test the preview features) and some must _not_ run with the preview features (in order to test that they are not accidentally supported) ? and most tests should pass in both. > The only practical solution to this problem would be to change the TCK > such that it automatically runs all the required tests affected by the > preview features in both configurations. Running all available tests > in both configurations makes no sense and is a complete waste of time > (e.g. why should one have to run all the interactive AWT/Swing or > printing tests two times, with and without preview features enabled if > they are not affected at all by these features?). I disagree; a compliant implementation should pass the tests that do not rely on preview features in both modes. What if an implementation, in preview mode, implemented completely different semantics for a standard feature? If you only had to pass the ?preview? tests in preview mode, such an implementation would appear compliant, even though this is grossly inconsistent with the spirit. > Finally, I do realize now that my proposal to use preview features at > least within the reference implementation itself, in order to gather > some experience with the new features, doesn't play very well with > certification. Also, if the JDK itself would make use of such > features, it would only run with "--enable-preview" which is probably > not what we want (unless we find a way to enable preview features > transparently for the JDK classes only). Right. That?s why we don?t do this. Its a shame that we lose one of our best channels for feedback here, but that?s how the cookie bounces. From aph at redhat.com Tue Mar 19 10:12:41 2019 From: aph at redhat.com (Andrew Haley) Date: Tue, 19 Mar 2019 10:12:41 +0000 Subject: Certification of Preview Features In-Reply-To: <0FCF5BC1-DA97-46AA-B665-17DDB8F20D1E@oracle.com> References: <0FCF5BC1-DA97-46AA-B665-17DDB8F20D1E@oracle.com> Message-ID: <58fdd16e-89c9-194d-6881-e8fa2722b918@redhat.com> On 3/18/19 5:01 PM, Brian Goetz wrote: > I disagree; a compliant implementation should pass the tests that do > not rely on preview features in both modes. What if an > implementation, in preview mode, implemented completely different > semantics for a standard feature? If you only had to pass the > ?preview? tests in preview mode, such an implementation would appear > compliant, even though this is grossly inconsistent with the spirit. Should pass, yes. But I read Volker's question in the spirit of practicality: we cannot test the JDK in a reasonable time in all possible modes. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From volker.simonis at gmail.com Tue Mar 19 10:48:46 2019 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 19 Mar 2019 11:48:46 +0100 Subject: Certification of Preview Features In-Reply-To: <58fdd16e-89c9-194d-6881-e8fa2722b918@redhat.com> References: <0FCF5BC1-DA97-46AA-B665-17DDB8F20D1E@oracle.com> <58fdd16e-89c9-194d-6881-e8fa2722b918@redhat.com> Message-ID: Just for reference I've uploaded the stripped picture here: http://cr.openjdk.java.net/~simonis/webrevs/2019/certi_preview.jpg On Tue, Mar 19, 2019 at 11:12 AM Andrew Haley wrote: > > On 3/18/19 5:01 PM, Brian Goetz wrote: > > > I disagree; a compliant implementation should pass the tests that do > > not rely on preview features in both modes. What if an > > implementation, in preview mode, implemented completely different > > semantics for a standard feature? If you only had to pass the > > ?preview? tests in preview mode, such an implementation would appear > > compliant, even though this is grossly inconsistent with the spirit. > > Should pass, yes. But I read Volker's question in the spirit of > practicality: we cannot test the JDK in a reasonable time in all > possible modes. > Yes, it is mostly about usability/practicality. Already now it takes an experienced engineer who is mostly doing certifications about two full days to do a complete, compliant certification. Now multiply this with the number of platforms and Java versions and you'll get a considerable effort. The "Preview Feature" testing in its current form will nearly double this effort one more time. The other thing is that already now it is quite complex to do a "compliant certification". And because nobody is allowed to share his certification results, it is impossible to judge if somebody who claims he has certified Java SE, really did a "compliant certification". We already have too many knobs and settings which have to be manually set in the correct way anyway. The new settings and instructions for the Preview features (as shown in the picture) further complicate certifications. What I was suggesting is that the JCK test harness should run a "reasonable amount" of tests in both modes automatically (without any interaction from the tester). "Reasonable amount" is a test set determined by the authors of the Preview Feature tests suite. Putting a little more effort into the creation of the TCK will save huge amounts of effort during certification and make the certification results more reliable. As a prominent example, I really see absolutely no reason in running the interactive AWT/Swing test (which require several hours of tedious, manual work) two times just because of the "Switch Expressions" preview feature. > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Tue Mar 19 11:08:28 2019 From: aph at redhat.com (Andrew Haley) Date: Tue, 19 Mar 2019 11:08:28 +0000 Subject: Certification of Preview Features In-Reply-To: References: <0FCF5BC1-DA97-46AA-B665-17DDB8F20D1E@oracle.com> <58fdd16e-89c9-194d-6881-e8fa2722b918@redhat.com> Message-ID: On 3/19/19 10:48 AM, Volker Simonis wrote: > The other thing is that already now it is quite complex to do a > "compliant certification". And because nobody is allowed to share his > certification results, it is impossible to judge if somebody who > claims he has certified Java SE, really did a "compliant > certification". We already have too many knobs and settings which have > to be manually set in the correct way anyway. The new settings and > instructions for the Preview features (as shown in the picture) > further complicate certifications. Well, yes, true. We already know that the TCK runs on the honour system, and a strict mathematical interpretation of correctness isn't possible. The point is that a Java implementation must pass the TCK in all modes: it is up to an implementer to run the tests as much as necessary to establish that beyond reasonable doubt. We cannot test all possibilities: it isn't even possible even to test exhaustively a 64*64-bit multiplication, we already know that. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From Tim_Ellison at uk.ibm.com Tue Mar 19 21:46:57 2019 From: Tim_Ellison at uk.ibm.com (Tim Ellison) Date: Wed, 20 Mar 2019 06:46:57 +0900 Subject: Certification of Preview Features In-Reply-To: References: <0FCF5BC1-DA97-46AA-B665-17DDB8F20D1E@oracle.com> <58fdd16e-89c9-194d-6881-e8fa2722b918@redhat.com> Message-ID: "Volker Simonis" wrote on 19/03/2019 19:48:46: > Just for reference I've uploaded the stripped picture here: > > https://urldefense.proofpoint.com/v2/url? > u=http-3A__cr.openjdk.java.net_-7Esimonis_webrevs_2019_certi-5Fpreview.jpg&d=DwIFaQ&c=jf_iaSHvJObTbx- > siA1ZOg&r=AtEh0PzjNUxLZESO8_yZ6DCI4DYOhKAFQsZXG3kYuYM&m=43zuj3fHvHrNSVl98AXrNTp2Yjh5Tenxf8mOOzzGKCY&s=i55Xf3ndRvtWkBbQPPvIa_fUY40THkVVz55WG6Fv-4M&e= > > On Tue, Mar 19, 2019 at 11:12 AM Andrew Haley wrote: > > > > On 3/18/19 5:01 PM, Brian Goetz wrote: > > > > > I disagree; a compliant implementation should pass the tests that do > > > not rely on preview features in both modes. What if an > > > implementation, in preview mode, implemented completely different > > > semantics for a standard feature? If you only had to pass the > > > ?preview? tests in preview mode, such an implementation would appear > > > compliant, even though this is grossly inconsistent with the spirit. > > > > Should pass, yes. But I read Volker's question in the spirit of > > practicality: we cannot test the JDK in a reasonable time in all > > possible modes. > > > > Yes, it is mostly about usability/practicality. Already now it takes > an experienced engineer who is mostly doing certifications about two > full days to do a complete, compliant certification. Now multiply this > with the number of platforms and Java versions and you'll get a > considerable effort. The "Preview Feature" testing in its current form > will nearly double this effort one more time. > > The other thing is that already now it is quite complex to do a > "compliant certification". And because nobody is allowed to share his > certification results, it is impossible to judge if somebody who > claims he has certified Java SE, really did a "compliant > certification". We already have too many knobs and settings which have > to be manually set in the correct way anyway. The new settings and > instructions for the Preview features (as shown in the picture) > further complicate certifications. > > What I was suggesting is that the JCK test harness should run a > "reasonable amount" of tests in both modes automatically (without any > interaction from the tester). "Reasonable amount" is a test set > determined by the authors of the Preview Feature tests suite. Putting > a little more effort into the creation of the TCK will save huge > amounts of effort during certification and make the certification > results more reliable. So there are (at least) two ways of looking at this: The JCK authors already make decisions about how extensively they are testing to the specification. Of course, compliance testing cannot be (mathematically) complete so the the JCK authors are using their skill and judgement to decide the number and scope of tests for particular APIs and features to determine spec compliance. It sounds like you are arguing that the same informed opinion of the JCK authors be used to decide the extent of testing under the Preview Features. However, where the JCK framework *can* be used to test exhaustively, such as via setting modes, it makes sense to allow the tests to be run that way without artificially limiting the tests that can be run in such modes. That is part of the spec compliance definition. It then becomes part of the specification compliance to require all tests pass in all modes, but for the JCK users to decide where they are confident in making the assertion of a pass. If somebody subsequently demonstrates that the user's confidence was misplaced, i.e. by demonstrating a failure mode, then that's a bug to be fixed. I suppose the analogy is: we declare releases complete when tested amongst a representative set of dependent libraries in the OS, and declare confidence that the release will work in other combinations of OS library versions etc without explicitly testing them. If somebody shows that is untrue we fix it. > As a prominent example, I really see absolutely no reason in running > the interactive AWT/Swing test (which require several hours of > tedious, manual work) two times just because of the "Switch > Expressions" preview feature. You might make the judgement call not to run those tests and still declare compliance; but if somebody can demonstrate that Switch Expressions actually does break the interactive tests then I'm sure you would decide to fix it. Regards, Tim Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU