From brian.goetz at oracle.com Tue Apr 2 14:57:03 2019 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 2 Apr 2019 10:57:03 -0400 Subject: Certification of Preview Features In-Reply-To: References: Message-ID: I have some additional guidance here. Hopefully this helps. The sole preview feature in Java SE 12 was switch expressions, which expand the Java language unconditionally. That means: 1. Of the programs that were legal in Java SE 11, all are legal in Java SE 12 whether preview features are disabled or enabled. 2. Of the programs that were illegal in Java SE 11, some are legal in Java SE 12 when preview features are enabled. (And some are still illegal in Java SE 12 whether preview features are disabled or enabled. For example, `switch (e) { case 1 ---> ...}` was illegal in Java SE 11 and is still illegal in Java SE 12 even with preview features enabled, because the arrow is written wrong.) #1 means that JCK 12 did not invalidate or invert pre-existing tests for non-preview features. #2 means that JCK 12 added tests for preview features (some positive, some negative) that require preview features to be enabled. Since the two sets of tests are orthogonal, you don't have to run the whole test suite twice to certify conformance. In the Configuration Editor, you can: 1. Disable preview features, and test only non-preview features. Test Platform -> Run tests on Preview Features -> "No" Standard Values... -> Keywords -> `!switch_expressions` 2. Enable preview features, and test only preview features. Test Platform -> Run tests on Preview Features -> "Yes" Standard Values... -> Keywords -> `switch_expressions` This incremental approach is already used for headless versus headful implementations. Every program that runs on a headless system will run on a headful system, and of the programs that won't run on a headless system, some will run on a headful system. Accordingly, the headless tests and the headful tests are orthogonal, and you don't have to run all of them twice to certify a headful implementation. The JCK harness lets you choose either the headless tests or the headless-plus-headful tests: in the Configuration Editor (Test Platform -> Headless System), you answer "No" to the interview question "Is the system you are testing headless?", and more tests will be executed than if you had answered "Yes". Since preview features are mandatory, you might wonder why the JCK harness doesn't run all the tests and have preview features enabled all the time. The answer is "an abundance of caution" -- JCK tries to avoid assumptions about how preview features are implemented. The concern is that when preview features are enabled, an implementation might give the wrong result for a test of a NON-preview feature. This won't happen for the OpenJDK implementation of switch expressions, but it could happen for a third-party implementation of another preview feature in future. That's why JCK 12 left it up to the implementer to determine if the whole suite can run with preview features enabled. In JCK 13, the User Guide will give more clarity on how to handle preview features' certification by using the keywords mentioned above. This should help avoid unnecessary runs when an implementation is a straightforward derivative of OpenJDK with no major changes to preview features' implementation. Looking forward, a preview feature might not expand the Java language unconditionally. JEP 12 notes the case of the `_` character, whereby the space of Java identifiers decreased between Java SE 8 and 9. Some future JCK N would need four keywords: (a) java_se_N_preview_language_features_positive_when_nopreview (b) java_se_N_preview_language_features_negative_when_nopreview (c) java_se_N_preview_language_features_positive_when_preview (d) java_se_N_preview_language_features_negative_when_preview In effect, JCK 12 added tests with the (c) and (d) keywords, both spelled `switch_expressions`. Support for "Drop _ as an identifier" would mean finding any pre-existing tests that exercise _ as an identifier, and adding the (a) or (b) keyword to each test as applicable; then copying all those tests and annotating the copies with the (d) keyword. (Longstanding JCK practice is that one test has one result, so if a test's result depends on the enablement of preview features, then what you need is two tests.) There would be no interview question about preview features at all -- the JCK harness would run every compiler test with either no (a)-(d) keywords, or just (a) or (b), with preview features disabled; and run every compiler test with just (c) or (d) with preview features enabled. > On Mar 18, 2019, at 11:59 AM, Volker Simonis wrote: > > Hi, > > we've recently started the TCK certification for Java SE 12. As we all > know, Java SE 12 is the first release with "Preview Features". As > we've previously discussed and as described in the "JEP 12: Preview > Language and VM Features" [1], preview features are fully specified > and fully implemented features of the Java SE platform which have to > be certified by the corresponding TCK. > > With the latest Java SE 12 TCK, this testing looks as follows (see > attached image): > > - one has to choose whether to run the test on the preview features or not > - the documentation mentions that one "may need to test the product in > both modes" > > First of all, "may need" is a much to weak requirement. According to > JEP 12 and our discussions around the topic I'm pretty sure that a > Java SE 12 compliant implementation will "HAVE TO" run the tests in > both modes. > > Second, the TCK doesn't mention which tests will have to be run in > "both modes". Running all the runtime and langtools tests (they have > the same "preview feature" switch) is a real big overhead. It actually > doubles the time for a correct certification (which already is quite > substantial). It also interferes with other test switches like > "verify/noverify" which already require to run a part of the tests in > two different configurations. Together with the preview features, we > will now theoretically get four configurations which will have to be > tested. > > The only practical solution to this problem would be to change the TCK > such that it automatically runs all the required tests affected by the > preview features in both configurations. Running all available tests > in both configurations makes no sense and is a complete waste of time > (e.g. why should one have to run all the interactive AWT/Swing or > printing tests two times, with and without preview features enabled if > they are not affected at all by these features?). > > Finally, I do realize now that my proposal to use preview features at > least within the reference implementation itself, in order to gather > some experience with the new features, doesn't play very well with > certification. Also, if the JDK itself would make use of such > features, it would only run with "--enable-preview" which is probably > not what we want (unless we find a way to enable preview features > transparently for the JDK classes only). > > What do you think? > > Regards, > Volker > > [1] http://openjdk.java.net/jeps/12 From volker.simonis at sap.com Tue Apr 2 16:15:01 2019 From: volker.simonis at sap.com (Simonis, Volker) Date: Tue, 2 Apr 2019 16:15:01 +0000 Subject: AW: Certification of Preview Features In-Reply-To: References: , Message-ID: Hi Brian, thanks a lot for looking into this issue. Your answer is definitely helpful! Although the envisioned solution with no interview questions about preview features at all would be of corse optimal, the interpretation you give for how to correctly run the tests as of today is pragmatic and a good compromise for us. Best regards, Volker ________________________________ Von: java-se-spec-experts im Auftrag von Brian Goetz Gesendet: Dienstag, 2. April 2019 16:57 An: Volker Simonis Cc: java-se-spec-experts at openjdk.java.net Betreff: Re: Certification of Preview Features I have some additional guidance here. Hopefully this helps. The sole preview feature in Java SE 12 was switch expressions, which expand the Java language unconditionally. That means: 1. Of the programs that were legal in Java SE 11, all are legal in Java SE 12 whether preview features are disabled or enabled. 2. Of the programs that were illegal in Java SE 11, some are legal in Java SE 12 when preview features are enabled. (And some are still illegal in Java SE 12 whether preview features are disabled or enabled. For example, `switch (e) { case 1 ---> ...}` was illegal in Java SE 11 and is still illegal in Java SE 12 even with preview features enabled, because the arrow is written wrong.) #1 means that JCK 12 did not invalidate or invert pre-existing tests for non-preview features. #2 means that JCK 12 added tests for preview features (some positive, some negative) that require preview features to be enabled. Since the two sets of tests are orthogonal, you don't have to run the whole test suite twice to certify conformance. In the Configuration Editor, you can: 1. Disable preview features, and test only non-preview features. Test Platform -> Run tests on Preview Features -> "No" Standard Values... -> Keywords -> `!switch_expressions` 2. Enable preview features, and test only preview features. Test Platform -> Run tests on Preview Features -> "Yes" Standard Values... -> Keywords -> `switch_expressions` This incremental approach is already used for headless versus headful implementations. Every program that runs on a headless system will run on a headful system, and of the programs that won't run on a headless system, some will run on a headful system. Accordingly, the headless tests and the headful tests are orthogonal, and you don't have to run all of them twice to certify a headful implementation. The JCK harness lets you choose either the headless tests or the headless-plus-headful tests: in the Configuration Editor (Test Platform -> Headless System), you answer "No" to the interview question "Is the system you are testing headless?", and more tests will be executed than if you had answered "Yes". Since preview features are mandatory, you might wonder why the JCK harness doesn't run all the tests and have preview features enabled all the time. The answer is "an abundance of caution" -- JCK tries to avoid assumptions about how preview features are implemented. The concern is that when preview features are enabled, an implementation might give the wrong result for a test of a NON-preview feature. This won't happen for the OpenJDK implementation of switch expressions, but it could happen for a third-party implementation of another preview feature in future. That's why JCK 12 left it up to the implementer to determine if the whole suite can run with preview features enabled. In JCK 13, the User Guide will give more clarity on how to handle preview features' certification by using the keywords mentioned above. This should help avoid unnecessary runs when an implementation is a straightforward derivative of OpenJDK with no major changes to preview features' implementation. Looking forward, a preview feature might not expand the Java language unconditionally. JEP 12 notes the case of the `_` character, whereby the space of Java identifiers decreased between Java SE 8 and 9. Some future JCK N would need four keywords: (a) java_se_N_preview_language_features_positive_when_nopreview (b) java_se_N_preview_language_features_negative_when_nopreview (c) java_se_N_preview_language_features_positive_when_preview (d) java_se_N_preview_language_features_negative_when_preview In effect, JCK 12 added tests with the (c) and (d) keywords, both spelled `switch_expressions`. Support for "Drop _ as an identifier" would mean finding any pre-existing tests that exercise _ as an identifier, and adding the (a) or (b) keyword to each test as applicable; then copying all those tests and annotating the copies with the (d) keyword. (Longstanding JCK practice is that one test has one result, so if a test's result depends on the enablement of preview features, then what you need is two tests.) There would be no interview question about preview features at all -- the JCK harness would run every compiler test with either no (a)-(d) keywords, or just (a) or (b), with preview features disabled; and run every compiler test with just (c) or (d) with preview features enabled. > On Mar 18, 2019, at 11:59 AM, Volker Simonis wrote: > > Hi, > > we've recently started the TCK certification for Java SE 12. As we all > know, Java SE 12 is the first release with "Preview Features". As > we've previously discussed and as described in the "JEP 12: Preview > Language and VM Features" [1], preview features are fully specified > and fully implemented features of the Java SE platform which have to > be certified by the corresponding TCK. > > With the latest Java SE 12 TCK, this testing looks as follows (see > attached image): > > - one has to choose whether to run the test on the preview features or not > - the documentation mentions that one "may need to test the product in > both modes" > > First of all, "may need" is a much to weak requirement. According to > JEP 12 and our discussions around the topic I'm pretty sure that a > Java SE 12 compliant implementation will "HAVE TO" run the tests in > both modes. > > Second, the TCK doesn't mention which tests will have to be run in > "both modes". Running all the runtime and langtools tests (they have > the same "preview feature" switch) is a real big overhead. It actually > doubles the time for a correct certification (which already is quite > substantial). It also interferes with other test switches like > "verify/noverify" which already require to run a part of the tests in > two different configurations. Together with the preview features, we > will now theoretically get four configurations which will have to be > tested. > > The only practical solution to this problem would be to change the TCK > such that it automatically runs all the required tests affected by the > preview features in both configurations. Running all available tests > in both configurations makes no sense and is a complete waste of time > (e.g. why should one have to run all the interactive AWT/Swing or > printing tests two times, with and without preview features enabled if > they are not affected at all by these features?). > > Finally, I do realize now that my proposal to use preview features at > least within the reference implementation itself, in order to gather > some experience with the new features, doesn't play very well with > certification. Also, if the JDK itself would make use of such > features, it would only run with "--enable-preview" which is probably > not what we want (unless we find a way to enable preview features > transparently for the JDK classes only). > > What do you think? > > Regards, > Volker > > [1] http://openjdk.java.net/jeps/12 From brian.goetz at oracle.com Wed Apr 10 19:02:02 2019 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 10 Apr 2019 15:02:02 -0400 Subject: Missed our meeting Message-ID: <83f8ee48-640a-a286-08d2-60bf3f78ddad@oracle.com> My apologies, our quarterly Monday EG call fell off my calendar. As far as I can tell, only Manoj noticed.? Which makes me ask -- did we need a meeting this month?? Should we reschedule, or roll forward to the next scheduled call?? Happy to do either, let me know. From aph at redhat.com Thu Apr 11 08:10:17 2019 From: aph at redhat.com (Andrew Haley) Date: Thu, 11 Apr 2019 09:10:17 +0100 Subject: Missed our meeting In-Reply-To: <83f8ee48-640a-a286-08d2-60bf3f78ddad@oracle.com> References: <83f8ee48-640a-a286-08d2-60bf3f78ddad@oracle.com> Message-ID: <51058bfb-0b3c-3650-e1f2-2960be46688c@redhat.com> On 4/10/19 8:02 PM, Brian Goetz wrote: > My apologies, our quarterly Monday EG call fell off my calendar. > > As far as I can tell, only Manoj noticed.? Which makes me ask -- did we > need a meeting this month?? Should we reschedule, or roll forward to the > next scheduled call?? Happy to do either, let me know. It fell off mine too. I'd like to refresh my mind about where we are up to. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From brian.goetz at oracle.com Thu Apr 11 15:19:51 2019 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 11 Apr 2019 11:19:51 -0400 Subject: Missed our meeting In-Reply-To: References: <83f8ee48-640a-a286-08d2-60bf3f78ddad@oracle.com> Message-ID: As you know, the dashboard includes both draft and candidate features -- but this is no guarantee that the feature will ever appear, or even that it is currently resourced.? "Candidate" means just that -- it's a candidate for a feature.? To turn a candidate into a feature requires a lot of work. We don't target ahead of time any more; we target when the delivery risk is below some (very low) epsilon.? What you are looking for is "which features might be done in 13?"? And I can't really answer that, because projecting "done" is an imprecise art (this is the the whole point of having moved to a time-boxed release strategy.) That said, it should be pretty obvious from watching the relevant mailing lists (e.g., amber-spec-experts) when a language feature is being actively worked on (you can see commits to the code) and when it is starting to stabilize (you will see spec discussions.) So the most important thing you can do is follow the discussions on the right lists.? (We also generally try to avoid targeting language features late in the release cycle, though there's no hard cutoff here.) Any feature that is previewed in one version must have some action taken in the next version, whether that action is an affirmative choice to re-preview (with or without changes), withdraw, or promote to permanent status.? These features should always be on your "I have to do something about" list.? "Expression switch" was a preview feature in 12, and there has been discussion on amber-spec-experts on making small changes to it. Language JEPs to which significant discussion has happened recently include: 348, 326, 305; you will want to familiarize yourself with what's going on there. Cheers, -Brian On 4/11/2019 12:18 AM, Manoj Palat wrote: > > Hi Brian, > I had just one query for the meeting to discuss - You can reply via > email itself. > > From the JEP Dashboard > , > I see a host of features in various stages for Java 13. I believe that > some of them may not make it for Java 13. Could you please share some > insights as to which would be the features which are more likely to > get into Java 13, especially those ones against "language" and > "javac", which are directly relevant for Eclipse Java Dev? > > [This would help us to prioritize given the limited resources we have > here.] > > Thanks, > Manoj/ > > Inactive hide details for Brian Goetz ---04/11/2019 12:34:23 AM---My > apologies, our quarterly Monday EG call fell off my calendBrian Goetz > ---04/11/2019 12:34:23 AM---My apologies, our quarterly Monday EG call > fell off my calendar. As far as I can tell, only Manoj no > > From: Brian Goetz > To: java-se-spec-experts at openjdk.java.net > Date: 04/11/2019 12:34 AM > Subject: Missed our meeting > Sent by: "java-se-spec-experts" > > > ------------------------------------------------------------------------ > > > > My apologies, our quarterly Monday EG call fell off my calendar. > > As far as I can tell, only Manoj noticed.? Which makes me ask -- did we > need a meeting this month?? Should we reschedule, or roll forward to the > next scheduled call?? Happy to do either, let me know. > > > >