From smarlow at redhat.com Tue Jan 5 20:35:18 2021 From: smarlow at redhat.com (Scott Marlow) Date: Tue, 5 Jan 2021 15:35:18 -0500 Subject: Newbie help needed getting setup to build SigTest + discussion about adding a exclude class by name option ... In-Reply-To: <0982f738-d912-6ed2-1c1b-305c300adccf@oracle.com> References: <812e16b0-af70-52a5-c8ab-b6440b97992c@redhat.com> <0982f738-d912-6ed2-1c1b-305c300adccf@oracle.com> Message-ID: <2ebed629-3c2d-596e-6fbb-7f54b4392156@redhat.com> Hi Victor, Thank you for the response! On 12/5/20 3:18 AM, victor.rudometov at oracle.com wrote: > Hi Scott, > > Thank you for your interest in SigTest. Please find my answers below. > > On 12/3/20 08:03, Scott Marlow wrote: >> Hi, >> >> I'm following the [1] guide to build locally but still have a few >> questions.? Is there a particular version of Apache Ant needed to >> build? ?I downloaded 1.9.15 as that should work with Java 7 but am >> getting a build failure [2]: >> ` >> src/com/sun/tdk/signaturetest/ant/ATest.java:165: error: cannot find >> symbol >> ??? [javac]???????? if (classpath == null || classpath.isEmpty()) { > > [1] is outdated and needs to be refreshed. > > I'm using ant 1.10.8 (older ones should work as well) with the > following minimal parameters: > > ant -f build/build.xml -Djdk7.home=/ws/java/java8 > -Djdk8.home=/ws/java/java8 -Djdk9.home=/ws/java/java11 > > Please use Java 8 for jdk7.home. There's probably a way to still use > 7, I have not checked it. > > You can use Java 9 or higher for jdk9.home, but I would recommend Java > 11 or higher, since 11 is LTS. > > You can use 15 or 16 too as jdk8.home and jdk9.home if features like > records or sealed types are present in your implementation. Sigtest > supports them. > Thanks, this helped me to build correctly! :) > >> I would like to explore a few ideas with SigTest related to [3] for >> the Jakarta EE Platform TCK which heavily depends on SigTest for SPEC >> API signature testing. Basically, we have users that would like to >> run our TCK against newer versions of Java SE than we have tested >> against. These users like the idea of being able to run the TCKs >> against the very latest Java SE versions.? Since our (Jakarta EE >> Platform TCK) is basically a Java application, it would be very >> useful to be able to run our Jakarta EE Platform EE TCKs against any >> Java SE version that is greater than our base requirement (currently >> we only support Java SE 8 but are working on Java SE 11 now). >> >> One idea is to create a SigTest pull request for adding an exclude >> class option so that the java.lang.Object public methods/fields will >> be excluded from signature generation/testing. > > I guess the functionality that you need is already there. You can > specify what should be tested by using -Package and > -PackageWithoutSubpackages options. > > You can also filter out packages (with subpackages) or individual > classes using -Exclude option. I tried excluding the `java` package during Setup which seemed to exclude most of the Java SE classes that I wanted to exclude from the generated signature, however then the `java` package classes are missing during the Test command. Using Java 8, I did the following with a recent local SigTest build: 1.? cd /tmp 2.? wget https://repo1.maven.org/maven2/jakarta/persistence/jakarta.persistence-api/3.0.0/jakarta.persistence-api-3.0.0.jar 3.? export SIGTEST=~/SIGTEST_BUILD/release/lib/ 4.? java -Xmx2g -jar $SIGTEST/sigtestdev.jar Setup -classpath jakarta.persistence-api-3.0.0.jar:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/jre/lib/jce.jar -filename /tmp/jakarta.persistence-api-3.0.0.sig -package jakarta -exclude java " Warning: the created signature file is not transitively-closed The signature file contains the following subclasses or subinterfaces of excluded java.io.Serializable: java.lang.Class, java.lang.String, java.lang.StackTraceElement, java.util.Calendar, java.util.Date, java.lang.Boolean, java.lang.Character, java.lang.Number, java.net.URL ... " On the Test command, I get then get failures : 5.? java -Xmx2g -jar $SIGTEST/sigtest.jar Test -Static -classpath jakarta.persistence-api-3.0.0.jar:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/jre/lib/jce.jar -filename /tmp/jakarta.persistence-api-3.0.0.sig -package jakarta " The following classes are required, but missing in the signature files:java.lang.Enum, java.lang.annotation.Repeatable, java.lang.annotation.Annotation, java.lang.annotation.Target, java.lang.annotation.Documented, java.lang.Object, java.lang.RuntimeException, java.lang.annotation.Retention STATUS:Error.Required classes not found! " > > If that is not flexible enough there is a possibility to create an > exclude plugin and set it via system property "exclude.plugin". Take a > look at com.sun.tdk.exclude.ExcludeList and -ExcludeSig option > allowing signature exclusions using regexps. Thanks, I will try using the Exclude extension next via com.sun.tdk.exclude.ExcludeList.? That does seem to be different than excluding the Java SE runtime classes from the generated signature files. I'm not really sure if the Exclude extension prevents the verification of: 1.? Jakarta EE SPEC API verification of super class (e.g. java.util.Map or java.lang.Object or some other Java SE runtime class that Jakarta EE SPEC API class "is a descendant of"). 2.? Jakarta EE SPEC API class must have a Java SE runtime class reference (as part of a public field/method). For example, if we are verifying that the super class of jakarta.persistence.Persistence (https://jakarta.ee/specifications/persistence/3.0/apidocs/jakarta.persistence/jakarta/persistence/persistence) class is java.lang.Object but find that the super class is some unexpected class like java.util.Map, the signature test must still fail. Scott > >> Imo, this should be a general option that allows a set of classes to >> be excluded (perhaps with some regular expression matching) and also >> allowing the command line exclude option to be specified multiple >> times (for excluding multiple classes by name). >> >> At least that is one idea.? My preference is to do this work upstream >> first [4].? If there is no interest in the pull request being merged, >> the next option could be forking the SigTest code into the >> jakartaee-tck-tools [5] and making the change there. Either option >> would be easy figure out once we have appropriate changes. > > Contributions are very much welcome. > > This mailing list is the best place to discuss new needed things in > SigTest. > > > Thanks. > > Victor. > > >> One other possible issue or question is whether this exclude class >> option would be a safe option to add to the OpenJDK SigTest tool? >> More specifically, could OpenJDK implementors use such an exclude >> option to cheat the JDK signature tests?? I'm not sure how that would >> happen but if that is a concern, please do speak up about that, so we >> can just proceed to do fork of SigTest. >> >> From a timing perspective, this `exclude class` change is not >> required at this point in our planning but it would be very very nice >> to have! >> >> Scott >> >> [1] https://wiki.openjdk.java.net/display/CodeTools/How+to+build >> [2] https://gist.github.com/scottmarlow/b48890c8c5365264a4e7ea5c44c0778c >> [3] https://github.com/eclipse-ee4j/jakartaee-tck/issues/156 >> [4] http://hg.openjdk.java.net/code-tools/sigtest >> [5] https://github.com/eclipse-ee4j/jakartaee-tck-tools >> > From victor.rudometov at oracle.com Tue Jan 5 22:33:54 2021 From: victor.rudometov at oracle.com (victor.rudometov at oracle.com) Date: Tue, 5 Jan 2021 14:33:54 -0800 Subject: Newbie help needed getting setup to build SigTest + discussion about adding a exclude class by name option ... In-Reply-To: <2ebed629-3c2d-596e-6fbb-7f54b4392156@redhat.com> References: <812e16b0-af70-52a5-c8ab-b6440b97992c@redhat.com> <0982f738-d912-6ed2-1c1b-305c300adccf@oracle.com> <2ebed629-3c2d-596e-6fbb-7f54b4392156@redhat.com> Message-ID: <329db1ca-785a-b661-8358-01dbe95e8df4@oracle.com> Hi Scott, I would eliminate -exclude java in your setup command making it: 4.? java -Xmx2g -jar $SIGTEST/sigtestdev.jar Setup -classpath jakarta.persistence-api-3.0.0.jar:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/jre/lib/jce.jar -filename /tmp/jakarta.persistence-api-3.0.0.sig -package jakarta So the file will store signatures of super classes and interfaces as well (from java* packages). When running the test -package jakarta option will limit testing to jakarta package and subpackages. It will check members and super classes (so no Map instead of Object is allowed). Thanks. Victor. On 1/5/21 12:35, Scott Marlow wrote: > > Hi Victor, > > Thank you for the response! > > On 12/5/20 3:18 AM, victor.rudometov at oracle.com wrote: >> Hi Scott, >> >> Thank you for your interest in SigTest. Please find my answers below. >> >> On 12/3/20 08:03, Scott Marlow wrote: >>> Hi, >>> >>> I'm following the [1] guide to build locally but still have a few >>> questions.? Is there a particular version of Apache Ant needed to >>> build? ?I downloaded 1.9.15 as that should work with Java 7 but am >>> getting a build failure [2]: >>> ` >>> src/com/sun/tdk/signaturetest/ant/ATest.java:165: error: cannot find >>> symbol >>> ??? [javac]???????? if (classpath == null || classpath.isEmpty()) { >> >> [1] is outdated and needs to be refreshed. >> >> I'm using ant 1.10.8 (older ones should work as well) with the >> following minimal parameters: >> >> ant -f build/build.xml -Djdk7.home=/ws/java/java8 >> -Djdk8.home=/ws/java/java8 -Djdk9.home=/ws/java/java11 >> >> Please use Java 8 for jdk7.home. There's probably a way to still use >> 7, I have not checked it. >> >> You can use Java 9 or higher for jdk9.home, but I would recommend >> Java 11 or higher, since 11 is LTS. >> >> You can use 15 or 16 too as jdk8.home and jdk9.home if features like >> records or sealed types are present in your implementation. Sigtest >> supports them. >> > > Thanks, this helped me to build correctly! :) > >> >>> I would like to explore a few ideas with SigTest related to [3] for >>> the Jakarta EE Platform TCK which heavily depends on SigTest for >>> SPEC API signature testing. Basically, we have users that would like >>> to run our TCK against newer versions of Java SE than we have tested >>> against. These users like the idea of being able to run the TCKs >>> against the very latest Java SE versions.? Since our (Jakarta EE >>> Platform TCK) is basically a Java application, it would be very >>> useful to be able to run our Jakarta EE Platform EE TCKs against any >>> Java SE version that is greater than our base requirement (currently >>> we only support Java SE 8 but are working on Java SE 11 now). >>> >>> One idea is to create a SigTest pull request for adding an exclude >>> class option so that the java.lang.Object public methods/fields will >>> be excluded from signature generation/testing. >> >> I guess the functionality that you need is already there. You can >> specify what should be tested by using -Package and >> -PackageWithoutSubpackages options. >> >> You can also filter out packages (with subpackages) or individual >> classes using -Exclude option. > > I tried excluding the `java` package during Setup which seemed to > exclude most of the Java SE classes that I wanted to exclude from the > generated signature, however then the `java` package classes are > missing during the Test command. > > Using Java 8, I did the following with a recent local SigTest build: > > 1.? cd /tmp > 2.? wget > https://repo1.maven.org/maven2/jakarta/persistence/jakarta.persistence-api/3.0.0/jakarta.persistence-api-3.0.0.jar > 3.? export SIGTEST=~/SIGTEST_BUILD/release/lib/ > 4.? java -Xmx2g -jar $SIGTEST/sigtestdev.jar Setup -classpath > jakarta.persistence-api-3.0.0.jar:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/jre/lib/jce.jar > -filename /tmp/jakarta.persistence-api-3.0.0.sig -package jakarta > -exclude java > > " > Warning: the created signature file is not transitively-closed > The signature file contains the following subclasses or subinterfaces > of excluded java.io.Serializable: > java.lang.Class, java.lang.String, java.lang.StackTraceElement, > java.util.Calendar, java.util.Date, java.lang.Boolean, > java.lang.Character, java.lang.Number, java.net.URL > ... > " > > On the Test command, I get then get failures : > > 5.? java -Xmx2g -jar $SIGTEST/sigtest.jar Test -Static -classpath > jakarta.persistence-api-3.0.0.jar:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/jre/lib/jce.jar > -filename /tmp/jakarta.persistence-api-3.0.0.sig -package jakarta > > " > The following classes are required, but missing in the signature > files:java.lang.Enum, java.lang.annotation.Repeatable, > java.lang.annotation.Annotation, java.lang.annotation.Target, > java.lang.annotation.Documented, java.lang.Object, > java.lang.RuntimeException, java.lang.annotation.Retention > STATUS:Error.Required classes not found! > " > > >> >> If that is not flexible enough there is a possibility to create an >> exclude plugin and set it via system property "exclude.plugin". Take >> a look at com.sun.tdk.exclude.ExcludeList and -ExcludeSig option >> allowing signature exclusions using regexps. > > Thanks, I will try using the Exclude extension next via > com.sun.tdk.exclude.ExcludeList.? That does seem to be different than > excluding the Java SE runtime classes from the generated signature files. > > I'm not really sure if the Exclude extension prevents the verification of: > > 1.? Jakarta EE SPEC API verification of super class (e.g. > java.util.Map or java.lang.Object or some other Java SE runtime class > that Jakarta EE SPEC API class "is a descendant of"). > > 2.? Jakarta EE SPEC API class must have a Java SE runtime class > reference (as part of a public field/method). > > For example, if we are verifying that the super class of > jakarta.persistence.Persistence > (https://jakarta.ee/specifications/persistence/3.0/apidocs/jakarta.persistence/jakarta/persistence/persistence) > class is java.lang.Object but find that the super class is some > unexpected class like java.util.Map, the signature test must still fail. > > Scott > >> >>> Imo, this should be a general option that allows a set of classes to >>> be excluded (perhaps with some regular expression matching) and also >>> allowing the command line exclude option to be specified multiple >>> times (for excluding multiple classes by name). >>> >>> At least that is one idea.? My preference is to do this work >>> upstream first [4].? If there is no interest in the pull request >>> being merged, the next option could be forking the SigTest code into >>> the jakartaee-tck-tools [5] and making the change there. Either >>> option would be easy figure out once we have appropriate changes. >> >> Contributions are very much welcome. >> >> This mailing list is the best place to discuss new needed things in >> SigTest. >> >> >> Thanks. >> >> Victor. >> >> >>> One other possible issue or question is whether this exclude class >>> option would be a safe option to add to the OpenJDK SigTest tool? >>> More specifically, could OpenJDK implementors use such an exclude >>> option to cheat the JDK signature tests?? I'm not sure how that >>> would happen but if that is a concern, please do speak up about >>> that, so we can just proceed to do fork of SigTest. >>> >>> From a timing perspective, this `exclude class` change is not >>> required at this point in our planning but it would be very very >>> nice to have! >>> >>> Scott >>> >>> [1] https://wiki.openjdk.java.net/display/CodeTools/How+to+build >>> [2] >>> https://gist.github.com/scottmarlow/b48890c8c5365264a4e7ea5c44c0778c >>> [3] https://github.com/eclipse-ee4j/jakartaee-tck/issues/156 >>> [4] http://hg.openjdk.java.net/code-tools/sigtest >>> [5] https://github.com/eclipse-ee4j/jakartaee-tck-tools >>> >> From smarlow at redhat.com Wed Jan 6 18:44:20 2021 From: smarlow at redhat.com (Scott Marlow) Date: Wed, 6 Jan 2021 13:44:20 -0500 Subject: Newbie help needed getting setup to build SigTest + discussion about adding a exclude class by name option ... In-Reply-To: <329db1ca-785a-b661-8358-01dbe95e8df4@oracle.com> References: <812e16b0-af70-52a5-c8ab-b6440b97992c@redhat.com> <0982f738-d912-6ed2-1c1b-305c300adccf@oracle.com> <2ebed629-3c2d-596e-6fbb-7f54b4392156@redhat.com> <329db1ca-785a-b661-8358-01dbe95e8df4@oracle.com> Message-ID: <9e8365fb-e285-3fb6-545e-0b777fe28bf7@redhat.com> Hi Victor, On 1/5/21 5:33 PM, victor.rudometov at oracle.com wrote: > Hi Scott, > > I would eliminate -exclude java in your setup command making it: > > 4.? java -Xmx2g -jar $SIGTEST/sigtestdev.jar Setup -classpath > jakarta.persistence-api-3.0.0.jar:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/jre/lib/jce.jar > -filename /tmp/jakarta.persistence-api-3.0.0.sig -package jakarta > > So the file will store signatures of super classes and interfaces as > well (from java* packages). > > When running the test -package jakarta option will limit testing to > jakarta package and subpackages. It will check members and super > classes (so no Map instead of Object is allowed). Thanks, I am trying this now.? It seems to be an improvement. However, when generating signatures on Java SE 8 but testing on Java SE 11, I cannot seem to ignore/exclude Deprecated annotation that changed in Java SE 9. The Java SE 8 signature for the jakarta.persistence.Persistence class contains: "? anno 0 java.lang.Deprecated() " The Java SE 11 signature for the jakarta.persistence.Persistence class contains: "anno 0 java.lang.Deprecated(boolean forRemoval=false, java.lang.String since="") " When running the test command on Java SE 11 with the signature generated under Java SE 8 I see: "Class jakarta.persistence.Persistence ? Missed Annotations ??? PERSISTENCE_PROVIDER:anno 0 java.lang.Deprecated() ??? providers:anno 0 java.lang.Deprecated() ? Added Annotations ??? PERSISTENCE_PROVIDER:anno 0 java.lang.Deprecated(boolean forRemoval=false, java.lang.String since="") ??? providers:anno 0 java.lang.Deprecated(boolean forRemoval=false, java.lang.String since="") ??? finalize():anno 0 java.lang.Deprecated(boolean forRemoval=false, java.lang.String since="9") " Is there any SigTest magic built in to ignore the Deprecated annotation changes? Thanks, Scott > > > Thanks. > > Victor. From smarlow at redhat.com Thu Jan 7 02:52:13 2021 From: smarlow at redhat.com (Scott Marlow) Date: Wed, 6 Jan 2021 21:52:13 -0500 Subject: Newbie help needed getting setup to build SigTest + discussion about adding a exclude class by name option ... In-Reply-To: <9e8365fb-e285-3fb6-545e-0b777fe28bf7@redhat.com> References: <812e16b0-af70-52a5-c8ab-b6440b97992c@redhat.com> <0982f738-d912-6ed2-1c1b-305c300adccf@oracle.com> <2ebed629-3c2d-596e-6fbb-7f54b4392156@redhat.com> <329db1ca-785a-b661-8358-01dbe95e8df4@oracle.com> <9e8365fb-e285-3fb6-545e-0b777fe28bf7@redhat.com> Message-ID: <08c37b1f-efa0-c236-3c00-e4d0e640b2de@redhat.com> Fyi, the underlying goal is to allow the Jakarta EE TCK signature tests to be run against newer Java SE versions than the base Java SE version used to generate the signatures. Some text from the jakartaee-tck/issues/156 tracking issue [1]: " The recorded signatures include every field and method in every class and interface in the API, as well as the information for every superclass or interface. A consequence of this is that it records the signatures of every JDK class used by the API. That means that a different signature file is needed to test an API on JDK 12 vs JDK 11 (for example), even though the API works identically on both. There's no way to say "this API requires JDK 11 or newer". " With regard to my previous email below asking about the Deprecated annotation checking, I am wondering if the two different Deprecated annotation signatures (Java SE 8 vs SE 11) should be considered compatible since either form (zero, one or two parameters) could be executed?? If the answer is that it could depend on the behaviour expected by the user of the SigTest tooling, perhaps an extension could be introduced. Scott [1] https://github.com/eclipse-ee4j/jakartaee-tck/issues/156 On 1/6/21 1:44 PM, Scott Marlow wrote: > > Hi Victor, > > On 1/5/21 5:33 PM, victor.rudometov at oracle.com wrote: >> Hi Scott, >> >> I would eliminate -exclude java in your setup command making it: >> >> 4.? java -Xmx2g -jar $SIGTEST/sigtestdev.jar Setup -classpath >> jakarta.persistence-api-3.0.0.jar:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/jre/lib/jce.jar >> -filename /tmp/jakarta.persistence-api-3.0.0.sig -package jakarta >> >> So the file will store signatures of super classes and interfaces as >> well (from java* packages). >> >> When running the test -package jakarta option will limit testing to >> jakarta package and subpackages. It will check members and super >> classes (so no Map instead of Object is allowed). > > > Thanks, I am trying this now.? It seems to be an improvement. > > However, when generating signatures on Java SE 8 but testing on Java > SE 11, I cannot seem to ignore/exclude Deprecated annotation that > changed in Java SE 9. > > The Java SE 8 signature for the jakarta.persistence.Persistence class > contains: > > "? anno 0 java.lang.Deprecated() > " > > The Java SE 11 signature for the jakarta.persistence.Persistence class > contains: > > "anno 0 java.lang.Deprecated(boolean forRemoval=false, > java.lang.String since="") > " > > When running the test command on Java SE 11 with the signature > generated under Java SE 8 I see: > > "Class jakarta.persistence.Persistence > ? Missed Annotations > ??? PERSISTENCE_PROVIDER:anno 0 java.lang.Deprecated() > ??? providers:anno 0 java.lang.Deprecated() > ? Added Annotations > ??? PERSISTENCE_PROVIDER:anno 0 java.lang.Deprecated(boolean > forRemoval=false, java.lang.String since="") > ??? providers:anno 0 java.lang.Deprecated(boolean forRemoval=false, > java.lang.String since="") > ??? finalize():anno 0 java.lang.Deprecated(boolean forRemoval=false, > java.lang.String since="9") > " > > Is there any SigTest magic built in to ignore the Deprecated > annotation changes? > > > Thanks, > Scott > >> >> >> Thanks. >> >> Victor. > > > From victor.rudometov at oracle.com Thu Jan 7 22:23:51 2021 From: victor.rudometov at oracle.com (victor.rudometov at oracle.com) Date: Thu, 7 Jan 2021 14:23:51 -0800 Subject: Newbie help needed getting setup to build SigTest + discussion about adding a exclude class by name option ... In-Reply-To: <08c37b1f-efa0-c236-3c00-e4d0e640b2de@redhat.com> References: <812e16b0-af70-52a5-c8ab-b6440b97992c@redhat.com> <0982f738-d912-6ed2-1c1b-305c300adccf@oracle.com> <2ebed629-3c2d-596e-6fbb-7f54b4392156@redhat.com> <329db1ca-785a-b661-8358-01dbe95e8df4@oracle.com> <9e8365fb-e285-3fb6-545e-0b777fe28bf7@redhat.com> <08c37b1f-efa0-c236-3c00-e4d0e640b2de@redhat.com> Message-ID: Hi Scott, Technically the signatures do change when we switch from 8 to 11 and annotations are either added to finalize() method or default empty parameters are added to Deprecated. Annotation changes can affect the behavior of reflective APIs that work with annotations, therefore Sigtest is correct to report an error. If we do not want to track these I see exclude plugin as one of possible options (through check() method there). Another one is to work through PluginAPI and either transform or filter the signature. Thanks, Victor. On 1/6/21 18:52, Scott Marlow wrote: > > Fyi, the underlying goal is to allow the Jakarta EE TCK signature > tests to be run against newer Java SE versions than the base Java SE > version used to generate the signatures. > > Some text from the jakartaee-tck/issues/156 tracking issue [1]: > " > > The recorded signatures include every field and method in every class and > interface in the API, as well as the information for every superclass or > interface. A consequence of this is that it records the signatures of > every JDK class used by the API. > > That means that a different signature file is needed to test an API on > JDK 12 vs JDK 11 (for example), even though the API works identically on > both. There's no way to say "this API requires JDK 11 or newer". > " > > With regard to my previous email below asking about the Deprecated > annotation checking, I am wondering if the two different Deprecated > annotation signatures (Java SE 8 vs SE 11) should be considered > compatible since either form (zero, one or two parameters) could be > executed?? If the answer is that it could depend on the behaviour > expected by the user of the SigTest tooling, perhaps an extension > could be introduced. > > Scott > > [1] https://github.com/eclipse-ee4j/jakartaee-tck/issues/156 > > On 1/6/21 1:44 PM, Scott Marlow wrote: >> >> Hi Victor, >> >> On 1/5/21 5:33 PM, victor.rudometov at oracle.com wrote: >>> Hi Scott, >>> >>> I would eliminate -exclude java in your setup command making it: >>> >>> 4.? java -Xmx2g -jar $SIGTEST/sigtestdev.jar Setup -classpath >>> jakarta.persistence-api-3.0.0.jar:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/jre/lib/jce.jar >>> -filename /tmp/jakarta.persistence-api-3.0.0.sig -package jakarta >>> >>> So the file will store signatures of super classes and interfaces as >>> well (from java* packages). >>> >>> When running the test -package jakarta option will limit testing to >>> jakarta package and subpackages. It will check members and super >>> classes (so no Map instead of Object is allowed). >> >> >> Thanks, I am trying this now.? It seems to be an improvement. >> >> However, when generating signatures on Java SE 8 but testing on Java >> SE 11, I cannot seem to ignore/exclude Deprecated annotation that >> changed in Java SE 9. >> >> The Java SE 8 signature for the jakarta.persistence.Persistence class >> contains: >> >> "? anno 0 java.lang.Deprecated() >> " >> >> The Java SE 11 signature for the jakarta.persistence.Persistence >> class contains: >> >> "anno 0 java.lang.Deprecated(boolean forRemoval=false, >> java.lang.String since="") >> " >> >> When running the test command on Java SE 11 with the signature >> generated under Java SE 8 I see: >> >> "Class jakarta.persistence.Persistence >> ? Missed Annotations >> ??? PERSISTENCE_PROVIDER:anno 0 java.lang.Deprecated() >> ??? providers:anno 0 java.lang.Deprecated() >> ? Added Annotations >> ??? PERSISTENCE_PROVIDER:anno 0 java.lang.Deprecated(boolean >> forRemoval=false, java.lang.String since="") >> ??? providers:anno 0 java.lang.Deprecated(boolean forRemoval=false, >> java.lang.String since="") >> ??? finalize():anno 0 java.lang.Deprecated(boolean forRemoval=false, >> java.lang.String since="9") >> " >> >> Is there any SigTest magic built in to ignore the Deprecated >> annotation changes? >> >> >> Thanks, >> Scott >> >>> >>> >>> Thanks. >>> >>> Victor. >> >> >> From smarlow at redhat.com Tue Jan 19 22:35:14 2021 From: smarlow at redhat.com (Scott Marlow) Date: Tue, 19 Jan 2021 17:35:14 -0500 Subject: Newbie help needed getting setup to build SigTest + discussion about adding a exclude class by name option ... In-Reply-To: References: <812e16b0-af70-52a5-c8ab-b6440b97992c@redhat.com> <0982f738-d912-6ed2-1c1b-305c300adccf@oracle.com> <2ebed629-3c2d-596e-6fbb-7f54b4392156@redhat.com> <329db1ca-785a-b661-8358-01dbe95e8df4@oracle.com> <9e8365fb-e285-3fb6-545e-0b777fe28bf7@redhat.com> <08c37b1f-efa0-c236-3c00-e4d0e640b2de@redhat.com> Message-ID: On 1/7/21 5:23 PM, victor.rudometov at oracle.com wrote: > Hi Scott, > > Technically the signatures do change when we switch from 8 to 11 and > annotations are either added to finalize() method or default empty > parameters are added to Deprecated. Annotation changes can affect the > behavior of reflective APIs that work with annotations, therefore > Sigtest is correct to report an error. > > If we do not want to track these I see exclude plugin as one of > possible options (through check() method there). Another one is to > work through PluginAPI and either transform or filter the signature. Thanks, I will try the exclude plugin check() method. Also, below is a list of what I am trying to accomplish (pending more discussion + feedback): 1.? Do verify that the super class name doesn't change but do not verify the super class contents (e.g. no JDK class content checking). 2.? Do verify that each annotation name is as expected but do not verify the actual annotation signature. 3.? Do verify each public class method/field signature. 4.? Do fail if any public class method/field are added (for no super-setting rule). 5.? Do fail if any public class method/field are removed (for no sub-setting rule). 6.? To be clear, any JDK class referenced by name in the Jakarta EE SPEC API class will still be verified to still reference the same class name Thanks, Scott > > Thanks, > > Victor. > > On 1/6/21 18:52, Scott Marlow wrote: >> >> Fyi, the underlying goal is to allow the Jakarta EE TCK signature >> tests to be run against newer Java SE versions than the base Java SE >> version used to generate the signatures. >> >> Some text from the jakartaee-tck/issues/156 tracking issue [1]: >> " >> >> The recorded signatures include every field and method in every class >> and >> interface in the API, as well as the information for every superclass or >> interface. A consequence of this is that it records the signatures of >> every JDK class used by the API. >> >> That means that a different signature file is needed to test an API on >> JDK 12 vs JDK 11 (for example), even though the API works identically on >> both. There's no way to say "this API requires JDK 11 or newer". >> " >> >> With regard to my previous email below asking about the Deprecated >> annotation checking, I am wondering if the two different Deprecated >> annotation signatures (Java SE 8 vs SE 11) should be considered >> compatible since either form (zero, one or two parameters) could be >> executed?? If the answer is that it could depend on the behaviour >> expected by the user of the SigTest tooling, perhaps an extension >> could be introduced. >> >> Scott >> >> [1] https://github.com/eclipse-ee4j/jakartaee-tck/issues/156 >> >> On 1/6/21 1:44 PM, Scott Marlow wrote: >>> >>> Hi Victor, >>> >>> On 1/5/21 5:33 PM, victor.rudometov at oracle.com wrote: >>>> Hi Scott, >>>> >>>> I would eliminate -exclude java in your setup command making it: >>>> >>>> 4.? java -Xmx2g -jar $SIGTEST/sigtestdev.jar Setup -classpath >>>> jakarta.persistence-api-3.0.0.jar:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/jre/lib/jce.jar >>>> -filename /tmp/jakarta.persistence-api-3.0.0.sig -package jakarta >>>> >>>> So the file will store signatures of super classes and interfaces >>>> as well (from java* packages). >>>> >>>> When running the test -package jakarta option will limit testing to >>>> jakarta package and subpackages. It will check members and super >>>> classes (so no Map instead of Object is allowed). >>> >>> >>> Thanks, I am trying this now.? It seems to be an improvement. >>> >>> However, when generating signatures on Java SE 8 but testing on Java >>> SE 11, I cannot seem to ignore/exclude Deprecated annotation that >>> changed in Java SE 9. >>> >>> The Java SE 8 signature for the jakarta.persistence.Persistence >>> class contains: >>> >>> "? anno 0 java.lang.Deprecated() >>> " >>> >>> The Java SE 11 signature for the jakarta.persistence.Persistence >>> class contains: >>> >>> "anno 0 java.lang.Deprecated(boolean forRemoval=false, >>> java.lang.String since="") >>> " >>> >>> When running the test command on Java SE 11 with the signature >>> generated under Java SE 8 I see: >>> >>> "Class jakarta.persistence.Persistence >>> ? Missed Annotations >>> ??? PERSISTENCE_PROVIDER:anno 0 java.lang.Deprecated() >>> ??? providers:anno 0 java.lang.Deprecated() >>> ? Added Annotations >>> ??? PERSISTENCE_PROVIDER:anno 0 java.lang.Deprecated(boolean >>> forRemoval=false, java.lang.String since="") >>> ??? providers:anno 0 java.lang.Deprecated(boolean forRemoval=false, >>> java.lang.String since="") >>> ??? finalize():anno 0 java.lang.Deprecated(boolean forRemoval=false, >>> java.lang.String since="9") >>> " >>> >>> Is there any SigTest magic built in to ignore the Deprecated >>> annotation changes? >>> >>> >>> Thanks, >>> Scott >>> >>>> >>>> >>>> Thanks. >>>> >>>> Victor. >>> >>> >>> > From smarlow at redhat.com Thu Jan 21 15:28:27 2021 From: smarlow at redhat.com (Scott Marlow) Date: Thu, 21 Jan 2021 10:28:27 -0500 Subject: Newbie help needed getting setup to build SigTest + discussion about adding a exclude class by name option ... In-Reply-To: References: <812e16b0-af70-52a5-c8ab-b6440b97992c@redhat.com> <0982f738-d912-6ed2-1c1b-305c300adccf@oracle.com> <2ebed629-3c2d-596e-6fbb-7f54b4392156@redhat.com> <329db1ca-785a-b661-8358-01dbe95e8df4@oracle.com> <9e8365fb-e285-3fb6-545e-0b777fe28bf7@redhat.com> <08c37b1f-efa0-c236-3c00-e4d0e640b2de@redhat.com> Message-ID: <07e27f12-b348-8598-3543-68d6a8229405@redhat.com> On 1/19/21 5:35 PM, Scott Marlow wrote: > > > On 1/7/21 5:23 PM, victor.rudometov at oracle.com wrote: >> Hi Scott, >> >> Technically the signatures do change when we switch from 8 to 11 and >> annotations are either added to finalize() method or default empty >> parameters are added to Deprecated. Annotation changes can affect the >> behavior of reflective APIs that work with annotations, therefore >> Sigtest is correct to report an error. >> >> If we do not want to track these I see exclude plugin as one of >> possible options (through check() method there). Another one is to >> work through PluginAPI and either transform or filter the signature. > > Thanks, I will try the exclude plugin check() method. > A.? I tried updating the (com.sun.tdk.signaturetest) DefaultExcludeList to exclude `java.lang.Deprecated` during signature testing (using our original recorded signatures that do include `java.lang.Deprecated`) but the `java.lang.Deprecated` are still being verified and complained about. https://gist.github.com/scottmarlow/e4e7d33088c42ca3433bde98e4ed1d1c shows the callstack of the invocation to my modified DefaultExcludeList and some debug prints of passed information. B.? I tried another hack to exclude everything during signature testing by having DefaultExcludeList#check always throw `ExcludeException` for all invocations and I still had various missing classes, added classes, missing super classes, Added Annotations reported as well (https://gist.github.com/scottmarlow/9cbcf5703d87f1779a1c73d5ef750d8e has output from that using the Jakarta EE Signature test tooling that uses SigTest). Should my second hack (B) that throws `ExcludeException` for all calls to Except#check, result in there being zero Signature Test failures? Note that for this testing, we are only passing in the `jakarta` package names to be checked (via our sig-test-pkg-list_se11.txt package list file). > Also, below is a list of what I am trying to accomplish (pending more > discussion + feedback): > > 1.? Do verify that the super class name doesn't change but do not > verify the super class contents (e.g. no JDK class content checking). > 2.? Do verify that each annotation name is as expected but do not > verify the actual annotation signature. > 3.? Do verify each public class method/field signature. > 4.? Do fail if any public class method/field are added (for no > super-setting rule). > 5.? Do fail if any public class method/field are removed (for no > sub-setting rule). > 6.? To be clear, any JDK class referenced by name in the Jakarta EE > SPEC API class will still be verified to still reference the same > class name > > Thanks, > > Scott > >> >> Thanks, >> >> Victor. >> >> On 1/6/21 18:52, Scott Marlow wrote: >>> >>> Fyi, the underlying goal is to allow the Jakarta EE TCK signature >>> tests to be run against newer Java SE versions than the base Java SE >>> version used to generate the signatures. >>> >>> Some text from the jakartaee-tck/issues/156 tracking issue [1]: >>> " >>> >>> The recorded signatures include every field and method in every >>> class and >>> interface in the API, as well as the information for every >>> superclass or >>> interface. A consequence of this is that it records the signatures of >>> every JDK class used by the API. >>> >>> That means that a different signature file is needed to test an API on >>> JDK 12 vs JDK 11 (for example), even though the API works >>> identically on >>> both. There's no way to say "this API requires JDK 11 or newer". >>> " >>> >>> With regard to my previous email below asking about the Deprecated >>> annotation checking, I am wondering if the two different Deprecated >>> annotation signatures (Java SE 8 vs SE 11) should be considered >>> compatible since either form (zero, one or two parameters) could be >>> executed?? If the answer is that it could depend on the behaviour >>> expected by the user of the SigTest tooling, perhaps an extension >>> could be introduced. >>> >>> Scott >>> >>> [1] https://github.com/eclipse-ee4j/jakartaee-tck/issues/156 >>> >>> On 1/6/21 1:44 PM, Scott Marlow wrote: >>>> >>>> Hi Victor, >>>> >>>> On 1/5/21 5:33 PM, victor.rudometov at oracle.com wrote: >>>>> Hi Scott, >>>>> >>>>> I would eliminate -exclude java in your setup command making it: >>>>> >>>>> 4.? java -Xmx2g -jar $SIGTEST/sigtestdev.jar Setup -classpath >>>>> jakarta.persistence-api-3.0.0.jar:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/jre/lib/jce.jar >>>>> -filename /tmp/jakarta.persistence-api-3.0.0.sig -package jakarta >>>>> >>>>> So the file will store signatures of super classes and interfaces >>>>> as well (from java* packages). >>>>> >>>>> When running the test -package jakarta option will limit testing >>>>> to jakarta package and subpackages. It will check members and >>>>> super classes (so no Map instead of Object is allowed). >>>> >>>> >>>> Thanks, I am trying this now.? It seems to be an improvement. >>>> >>>> However, when generating signatures on Java SE 8 but testing on >>>> Java SE 11, I cannot seem to ignore/exclude Deprecated annotation >>>> that changed in Java SE 9. >>>> >>>> The Java SE 8 signature for the jakarta.persistence.Persistence >>>> class contains: >>>> >>>> "? anno 0 java.lang.Deprecated() >>>> " >>>> >>>> The Java SE 11 signature for the jakarta.persistence.Persistence >>>> class contains: >>>> >>>> "anno 0 java.lang.Deprecated(boolean forRemoval=false, >>>> java.lang.String since="") >>>> " >>>> >>>> When running the test command on Java SE 11 with the signature >>>> generated under Java SE 8 I see: >>>> >>>> "Class jakarta.persistence.Persistence >>>> ? Missed Annotations >>>> ??? PERSISTENCE_PROVIDER:anno 0 java.lang.Deprecated() >>>> ??? providers:anno 0 java.lang.Deprecated() >>>> ? Added Annotations >>>> ??? PERSISTENCE_PROVIDER:anno 0 java.lang.Deprecated(boolean >>>> forRemoval=false, java.lang.String since="") >>>> ??? providers:anno 0 java.lang.Deprecated(boolean forRemoval=false, >>>> java.lang.String since="") >>>> ??? finalize():anno 0 java.lang.Deprecated(boolean >>>> forRemoval=false, java.lang.String since="9") >>>> " >>>> >>>> Is there any SigTest magic built in to ignore the Deprecated >>>> annotation changes? >>>> >>>> >>>> Thanks, >>>> Scott >>>> >>>>> >>>>> >>>>> Thanks. >>>>> >>>>> Victor. >>>> >>>> >>>> >> From smarlow at redhat.com Tue Jan 26 17:21:35 2021 From: smarlow at redhat.com (Scott Marlow) Date: Tue, 26 Jan 2021 12:21:35 -0500 Subject: Newbie help needed getting setup to build SigTest + discussion about adding a exclude class by name option ... In-Reply-To: <07e27f12-b348-8598-3543-68d6a8229405@redhat.com> References: <812e16b0-af70-52a5-c8ab-b6440b97992c@redhat.com> <0982f738-d912-6ed2-1c1b-305c300adccf@oracle.com> <2ebed629-3c2d-596e-6fbb-7f54b4392156@redhat.com> <329db1ca-785a-b661-8358-01dbe95e8df4@oracle.com> <9e8365fb-e285-3fb6-545e-0b777fe28bf7@redhat.com> <08c37b1f-efa0-c236-3c00-e4d0e640b2de@redhat.com> <07e27f12-b348-8598-3543-68d6a8229405@redhat.com> Message-ID: <4aa9df21-7c55-1e26-ccaa-14e71260af53@redhat.com> Hi, https://gist.github.com/scottmarlow/0cebe942116fc1d3aef38c61b09e3395 shows the `java.lang.ClassNotFoundException: java.util.EventObject` that is thrown due to the "package list" not being checked (I think) by com.sun.tdk.signaturetest.core.MemberCollectionBuilder.createMembers. I am hacking some local changes via https://github.com/scottmarlow/sigtest/tree/explore_more_changes to try some changes. Would it make sense create a PackageChecker interface that could be passed to MemberCollectionBuilder#createMembers so that MemberCollectionBuilder would have the "package list" and know not to reference classes that are not on the "package list"? Are github pull requests accepted? Scott On 1/21/21 10:28 AM, Scott Marlow wrote: > > > On 1/19/21 5:35 PM, Scott Marlow wrote: >> >> >> On 1/7/21 5:23 PM, victor.rudometov at oracle.com wrote: >>> Hi Scott, >>> >>> Technically the signatures do change when we switch from 8 to 11 and >>> annotations are either added to finalize() method or default empty >>> parameters are added to Deprecated. Annotation changes can affect >>> the behavior of reflective APIs that work with annotations, >>> therefore Sigtest is correct to report an error. >>> >>> If we do not want to track these I see exclude plugin as one of >>> possible options (through check() method there). Another one is to >>> work through PluginAPI and either transform or filter the signature. >> >> Thanks, I will try the exclude plugin check() method. >> > A.? I tried updating the (com.sun.tdk.signaturetest) > DefaultExcludeList to exclude `java.lang.Deprecated` during signature > testing (using our original recorded signatures that do include > `java.lang.Deprecated`) but the `java.lang.Deprecated` are still being > verified and complained about. > https://gist.github.com/scottmarlow/e4e7d33088c42ca3433bde98e4ed1d1c > shows the callstack of the invocation to my modified > DefaultExcludeList and some debug prints of passed information. > > B.? I tried another hack to exclude everything during signature > testing by having DefaultExcludeList#check always throw > `ExcludeException` for all invocations and I still had various missing > classes, added classes, missing super classes, Added Annotations > reported as well > (https://gist.github.com/scottmarlow/9cbcf5703d87f1779a1c73d5ef750d8e > has output from that using the Jakarta EE Signature test tooling that > uses SigTest). > > Should my second hack (B) that throws `ExcludeException` for all calls > to Except#check, result in there being zero Signature Test failures? > > Note that for this testing, we are only passing in the `jakarta` > package names to be checked (via our sig-test-pkg-list_se11.txt > package list file). > >> Also, below is a list of what I am trying to accomplish (pending more >> discussion + feedback): >> >> 1.? Do verify that the super class name doesn't change but do not >> verify the super class contents (e.g. no JDK class content checking). >> 2.? Do verify that each annotation name is as expected but do not >> verify the actual annotation signature. >> 3.? Do verify each public class method/field signature. >> 4.? Do fail if any public class method/field are added (for no >> super-setting rule). >> 5.? Do fail if any public class method/field are removed (for no >> sub-setting rule). >> 6.? To be clear, any JDK class referenced by name in the Jakarta EE >> SPEC API class will still be verified to still reference the same >> class name >> >> Thanks, >> >> Scott >> >>> >>> Thanks, >>> >>> Victor. >>> >>> On 1/6/21 18:52, Scott Marlow wrote: >>>> >>>> Fyi, the underlying goal is to allow the Jakarta EE TCK signature >>>> tests to be run against newer Java SE versions than the base Java >>>> SE version used to generate the signatures. >>>> >>>> Some text from the jakartaee-tck/issues/156 tracking issue [1]: >>>> " >>>> >>>> The recorded signatures include every field and method in every >>>> class and >>>> interface in the API, as well as the information for every >>>> superclass or >>>> interface. A consequence of this is that it records the signatures of >>>> every JDK class used by the API. >>>> >>>> That means that a different signature file is needed to test an API on >>>> JDK 12 vs JDK 11 (for example), even though the API works >>>> identically on >>>> both. There's no way to say "this API requires JDK 11 or newer". >>>> " >>>> >>>> With regard to my previous email below asking about the Deprecated >>>> annotation checking, I am wondering if the two different Deprecated >>>> annotation signatures (Java SE 8 vs SE 11) should be considered >>>> compatible since either form (zero, one or two parameters) could be >>>> executed?? If the answer is that it could depend on the behaviour >>>> expected by the user of the SigTest tooling, perhaps an extension >>>> could be introduced. >>>> >>>> Scott >>>> >>>> [1] https://github.com/eclipse-ee4j/jakartaee-tck/issues/156 >>>> >>>> On 1/6/21 1:44 PM, Scott Marlow wrote: >>>>> >>>>> Hi Victor, >>>>> >>>>> On 1/5/21 5:33 PM, victor.rudometov at oracle.com wrote: >>>>>> Hi Scott, >>>>>> >>>>>> I would eliminate -exclude java in your setup command making it: >>>>>> >>>>>> 4.? java -Xmx2g -jar $SIGTEST/sigtestdev.jar Setup -classpath >>>>>> jakarta.persistence-api-3.0.0.jar:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/jre/lib/jce.jar >>>>>> -filename /tmp/jakarta.persistence-api-3.0.0.sig -package jakarta >>>>>> >>>>>> So the file will store signatures of super classes and interfaces >>>>>> as well (from java* packages). >>>>>> >>>>>> When running the test -package jakarta option will limit testing >>>>>> to jakarta package and subpackages. It will check members and >>>>>> super classes (so no Map instead of Object is allowed). >>>>> >>>>> >>>>> Thanks, I am trying this now.? It seems to be an improvement. >>>>> >>>>> However, when generating signatures on Java SE 8 but testing on >>>>> Java SE 11, I cannot seem to ignore/exclude Deprecated annotation >>>>> that changed in Java SE 9. >>>>> >>>>> The Java SE 8 signature for the jakarta.persistence.Persistence >>>>> class contains: >>>>> >>>>> "? anno 0 java.lang.Deprecated() >>>>> " >>>>> >>>>> The Java SE 11 signature for the jakarta.persistence.Persistence >>>>> class contains: >>>>> >>>>> "anno 0 java.lang.Deprecated(boolean forRemoval=false, >>>>> java.lang.String since="") >>>>> " >>>>> >>>>> When running the test command on Java SE 11 with the signature >>>>> generated under Java SE 8 I see: >>>>> >>>>> "Class jakarta.persistence.Persistence >>>>> ? Missed Annotations >>>>> ??? PERSISTENCE_PROVIDER:anno 0 java.lang.Deprecated() >>>>> ??? providers:anno 0 java.lang.Deprecated() >>>>> ? Added Annotations >>>>> ??? PERSISTENCE_PROVIDER:anno 0 java.lang.Deprecated(boolean >>>>> forRemoval=false, java.lang.String since="") >>>>> ??? providers:anno 0 java.lang.Deprecated(boolean >>>>> forRemoval=false, java.lang.String since="") >>>>> ??? finalize():anno 0 java.lang.Deprecated(boolean >>>>> forRemoval=false, java.lang.String since="9") >>>>> " >>>>> >>>>> Is there any SigTest magic built in to ignore the Deprecated >>>>> annotation changes? >>>>> >>>>> >>>>> Thanks, >>>>> Scott >>>>> >>>>>> >>>>>> >>>>>> Thanks. >>>>>> >>>>>> Victor. >>>>> >>>>> >>>>> >>> From smarlow at redhat.com Wed Jan 27 14:49:28 2021 From: smarlow at redhat.com (Scott Marlow) Date: Wed, 27 Jan 2021 09:49:28 -0500 Subject: Newbie help needed getting setup to build SigTest + discussion about adding a exclude class by name option ... In-Reply-To: <4aa9df21-7c55-1e26-ccaa-14e71260af53@redhat.com> References: <812e16b0-af70-52a5-c8ab-b6440b97992c@redhat.com> <0982f738-d912-6ed2-1c1b-305c300adccf@oracle.com> <2ebed629-3c2d-596e-6fbb-7f54b4392156@redhat.com> <329db1ca-785a-b661-8358-01dbe95e8df4@oracle.com> <9e8365fb-e285-3fb6-545e-0b777fe28bf7@redhat.com> <08c37b1f-efa0-c236-3c00-e4d0e640b2de@redhat.com> <07e27f12-b348-8598-3543-68d6a8229405@redhat.com> <4aa9df21-7c55-1e26-ccaa-14e71260af53@redhat.com> Message-ID: On 1/26/21 12:21 PM, Scott Marlow wrote: > > Hi, > > https://gist.github.com/scottmarlow/0cebe942116fc1d3aef38c61b09e3395 > shows the `java.lang.ClassNotFoundException: java.util.EventObject` > that is thrown due to the "package list" not being checked (I think) > by com.sun.tdk.signaturetest.core.MemberCollectionBuilder.createMembers. > I added another example call stack to the end of https://gist.github.com/scottmarlow/0cebe942116fc1d3aef38c61b09e3395 that shows a need for passing a PackageChecker to ClassHierarchyImpl so that classes that are not on the "package list" or are excluded are not referenced. I will try hard coding some class packages to ignore in ClassHierarchyImpl + MemberCollectionBuilder so that I can make progress with proving that a change is possible (even if done via code changes that will never get merged in).? If I can demonstrate that it is helpful to be able to exclude certain classes from ClassHierarchyImpl + MemberCollectionBuilder, we will then need to consider if it makes sense for SigTest to be adapted to more deeply respect the package/exclude lists. I am not surprised that with my hard coding of the exclude list to ignore more Java SE classes, that I am seeing a "APICheck 1" test failure due to "Missing Methods".? I will see if I can disable tests that fail for now (not a good long term idea and not something I would include in a pull request). :-) Scott > I am hacking some local changes via > https://github.com/scottmarlow/sigtest/tree/explore_more_changes to > try some changes. > > Would it make sense create a PackageChecker interface that could be > passed to MemberCollectionBuilder#createMembers so that > MemberCollectionBuilder would have the "package list" and know not to > reference classes that are not on the "package list"? > > Are github pull requests accepted? > > Scott > > On 1/21/21 10:28 AM, Scott Marlow wrote: >> >> >> On 1/19/21 5:35 PM, Scott Marlow wrote: >>> >>> >>> On 1/7/21 5:23 PM, victor.rudometov at oracle.com wrote: >>>> Hi Scott, >>>> >>>> Technically the signatures do change when we switch from 8 to 11 >>>> and annotations are either added to finalize() method or default >>>> empty parameters are added to Deprecated. Annotation changes can >>>> affect the behavior of reflective APIs that work with annotations, >>>> therefore Sigtest is correct to report an error. >>>> >>>> If we do not want to track these I see exclude plugin as one of >>>> possible options (through check() method there). Another one is to >>>> work through PluginAPI and either transform or filter the signature. >>> >>> Thanks, I will try the exclude plugin check() method. >>> >> A.? I tried updating the (com.sun.tdk.signaturetest) >> DefaultExcludeList to exclude `java.lang.Deprecated` during signature >> testing (using our original recorded signatures that do include >> `java.lang.Deprecated`) but the `java.lang.Deprecated` are still >> being verified and complained about. >> https://gist.github.com/scottmarlow/e4e7d33088c42ca3433bde98e4ed1d1c >> shows the callstack of the invocation to my modified >> DefaultExcludeList and some debug prints of passed information. >> >> B.? I tried another hack to exclude everything during signature >> testing by having DefaultExcludeList#check always throw >> `ExcludeException` for all invocations and I still had various >> missing classes, added classes, missing super classes, Added >> Annotations reported as well >> (https://gist.github.com/scottmarlow/9cbcf5703d87f1779a1c73d5ef750d8e >> has output from that using the Jakarta EE Signature test tooling that >> uses SigTest). >> >> Should my second hack (B) that throws `ExcludeException` for all >> calls to Except#check, result in there being zero Signature Test >> failures? >> >> Note that for this testing, we are only passing in the `jakarta` >> package names to be checked (via our sig-test-pkg-list_se11.txt >> package list file). >> >>> Also, below is a list of what I am trying to accomplish (pending >>> more discussion + feedback): >>> >>> 1.? Do verify that the super class name doesn't change but do not >>> verify the super class contents (e.g. no JDK class content checking). >>> 2.? Do verify that each annotation name is as expected but do not >>> verify the actual annotation signature. >>> 3.? Do verify each public class method/field signature. >>> 4.? Do fail if any public class method/field are added (for no >>> super-setting rule). >>> 5.? Do fail if any public class method/field are removed (for no >>> sub-setting rule). >>> 6.? To be clear, any JDK class referenced by name in the Jakarta EE >>> SPEC API class will still be verified to still reference the same >>> class name >>> >>> Thanks, >>> >>> Scott >>> >>>> >>>> Thanks, >>>> >>>> Victor. >>>> >>>> On 1/6/21 18:52, Scott Marlow wrote: >>>>> >>>>> Fyi, the underlying goal is to allow the Jakarta EE TCK signature >>>>> tests to be run against newer Java SE versions than the base Java >>>>> SE version used to generate the signatures. >>>>> >>>>> Some text from the jakartaee-tck/issues/156 tracking issue [1]: >>>>> " >>>>> >>>>> The recorded signatures include every field and method in every >>>>> class and >>>>> interface in the API, as well as the information for every >>>>> superclass or >>>>> interface. A consequence of this is that it records the signatures of >>>>> every JDK class used by the API. >>>>> >>>>> That means that a different signature file is needed to test an >>>>> API on >>>>> JDK 12 vs JDK 11 (for example), even though the API works >>>>> identically on >>>>> both. There's no way to say "this API requires JDK 11 or newer". >>>>> " >>>>> >>>>> With regard to my previous email below asking about the Deprecated >>>>> annotation checking, I am wondering if the two different >>>>> Deprecated annotation signatures (Java SE 8 vs SE 11) should be >>>>> considered compatible since either form (zero, one or two >>>>> parameters) could be executed?? If the answer is that it could >>>>> depend on the behaviour expected by the user of the SigTest >>>>> tooling, perhaps an extension could be introduced. >>>>> >>>>> Scott >>>>> >>>>> [1] https://github.com/eclipse-ee4j/jakartaee-tck/issues/156 >>>>> >>>>> On 1/6/21 1:44 PM, Scott Marlow wrote: >>>>>> >>>>>> Hi Victor, >>>>>> >>>>>> On 1/5/21 5:33 PM, victor.rudometov at oracle.com wrote: >>>>>>> Hi Scott, >>>>>>> >>>>>>> I would eliminate -exclude java in your setup command making it: >>>>>>> >>>>>>> 4.? java -Xmx2g -jar $SIGTEST/sigtestdev.jar Setup -classpath >>>>>>> jakarta.persistence-api-3.0.0.jar:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/jre/lib/jce.jar >>>>>>> -filename /tmp/jakarta.persistence-api-3.0.0.sig -package jakarta >>>>>>> >>>>>>> So the file will store signatures of super classes and >>>>>>> interfaces as well (from java* packages). >>>>>>> >>>>>>> When running the test -package jakarta option will limit testing >>>>>>> to jakarta package and subpackages. It will check members and >>>>>>> super classes (so no Map instead of Object is allowed). >>>>>> >>>>>> >>>>>> Thanks, I am trying this now.? It seems to be an improvement. >>>>>> >>>>>> However, when generating signatures on Java SE 8 but testing on >>>>>> Java SE 11, I cannot seem to ignore/exclude Deprecated annotation >>>>>> that changed in Java SE 9. >>>>>> >>>>>> The Java SE 8 signature for the jakarta.persistence.Persistence >>>>>> class contains: >>>>>> >>>>>> "? anno 0 java.lang.Deprecated() >>>>>> " >>>>>> >>>>>> The Java SE 11 signature for the jakarta.persistence.Persistence >>>>>> class contains: >>>>>> >>>>>> "anno 0 java.lang.Deprecated(boolean forRemoval=false, >>>>>> java.lang.String since="") >>>>>> " >>>>>> >>>>>> When running the test command on Java SE 11 with the signature >>>>>> generated under Java SE 8 I see: >>>>>> >>>>>> "Class jakarta.persistence.Persistence >>>>>> ? Missed Annotations >>>>>> ??? PERSISTENCE_PROVIDER:anno 0 java.lang.Deprecated() >>>>>> ??? providers:anno 0 java.lang.Deprecated() >>>>>> ? Added Annotations >>>>>> ??? PERSISTENCE_PROVIDER:anno 0 java.lang.Deprecated(boolean >>>>>> forRemoval=false, java.lang.String since="") >>>>>> ??? providers:anno 0 java.lang.Deprecated(boolean >>>>>> forRemoval=false, java.lang.String since="") >>>>>> ??? finalize():anno 0 java.lang.Deprecated(boolean >>>>>> forRemoval=false, java.lang.String since="9") >>>>>> " >>>>>> >>>>>> Is there any SigTest magic built in to ignore the Deprecated >>>>>> annotation changes? >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Scott >>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> Victor. >>>>>> >>>>>> >>>>>> >>>> From smarlow at redhat.com Thu Jan 28 19:25:56 2021 From: smarlow at redhat.com (Scott Marlow) Date: Thu, 28 Jan 2021 14:25:56 -0500 Subject: Additional use of the exclude/package list for Jakarta EE TCK signature checking... Message-ID: <1fe8ffc9-ca1d-fe96-cdf9-ea675e5a961e@redhat.com> Hi, I would like to make additional use of the exclude/package list (via SigTest#isPackageMember check or something like that) in the SignatureTest (or optionally add JakartaSignatureTest class).? I would also like to make additional use of the exclude/package list in classes { ClassHierarchyImpl, MemberCollectionBuilder } as shown in sigtest pull request [1]. As mentioned my previous (newbie) email thread, this is to help the Jakarta EE Platform TCKs to be able to pass on newer Java SE versions than the base Java SE version that a Jakarta EE release is targeted for.? This issue is described in more detail of the jakartaee-tck issue 156 [2]. Questions: 1. Could the JDK SigTest project still work correctly (for JDK project use) if SigTest#isPackageMember is called from ClassHierarchyImpl#findSuperclasses, MemberCollectionBuilder#addInherited, MemberCollectionBuilder#addInheritedFromInterfaces in the same places you can see the "java" + "javax" package checks hacked in [1]? 2. Would it make sense to add a new (Package) interface that implements SigTest#isPackageMember that an instance of could be made available to the above mentioned calls in #1? 3. Would it make sense to add a new option to enable the changes mentioned in #1 and such option would be disabled by default? Scott [1] https://github.com/openjdk/sigtest/pull/2 [2] https://github.com/eclipse-ee4j/jakartaee-tck/issues/156