Newbie help needed getting setup to build SigTest + discussion about adding a exclude class by name option ...

Scott Marlow smarlow at redhat.com
Wed Jan 27 14:49:28 UTC 2021 

On 1/26/21 12:21 PM, Scott Marlow wrote:
>
> Hi,
>
> https://gist.github.com/scottmarlow/0cebe942116fc1d3aef38c61b09e3395 
> shows the `java.lang.ClassNotFoundException: java.util.EventObject` 
> that is thrown due to the "package list" not being checked (I think) 
> by com.sun.tdk.signaturetest.core.MemberCollectionBuilder.createMembers.
>

I added another example call stack to the end of 
https://gist.github.com/scottmarlow/0cebe942116fc1d3aef38c61b09e3395 
that shows a need for passing a PackageChecker to ClassHierarchyImpl so 
that classes that are not on the "package list" or are excluded are not 
referenced.

I will try hard coding some class packages to ignore in 
ClassHierarchyImpl + MemberCollectionBuilder so that I can make progress 
with proving that a change is possible (even if done via code changes 
that will never get merged in).  If I can demonstrate that it is helpful 
to be able to exclude certain classes from ClassHierarchyImpl + 
MemberCollectionBuilder, we will then need to consider if it makes sense 
for SigTest to be adapted to more deeply respect the package/exclude lists.

I am not surprised that with my hard coding of the exclude list to 
ignore more Java SE classes, that I am seeing a "APICheck 1" test 
failure due to "Missing Methods".  I will see if I can disable tests 
that fail for now (not a good long term idea and not something I would 
include in a pull request). :-)

Scott


> I am hacking some local changes via 
> https://github.com/scottmarlow/sigtest/tree/explore_more_changes to 
> try some changes.
>
> Would it make sense create a PackageChecker interface that could be 
> passed to MemberCollectionBuilder#createMembers so that 
> MemberCollectionBuilder would have the "package list" and know not to 
> reference classes that are not on the "package list"?
>
> Are github pull requests accepted?
>
> Scott
>
> On 1/21/21 10:28 AM, Scott Marlow wrote:
>>
>>
>> On 1/19/21 5:35 PM, Scott Marlow wrote:
>>>
>>>
>>> On 1/7/21 5:23 PM, victor.rudometov at oracle.com wrote:
>>>> Hi Scott,
>>>>
>>>> Technically the signatures do change when we switch from 8 to 11 
>>>> and annotations are either added to finalize() method or default 
>>>> empty parameters are added to Deprecated. Annotation changes can 
>>>> affect the behavior of reflective APIs that work with annotations, 
>>>> therefore Sigtest is correct to report an error.
>>>>
>>>> If we do not want to track these I see exclude plugin as one of 
>>>> possible options (through check() method there). Another one is to 
>>>> work through PluginAPI and either transform or filter the signature.
>>>
>>> Thanks, I will try the exclude plugin check() method.
>>>
>> A.  I tried updating the (com.sun.tdk.signaturetest) 
>> DefaultExcludeList to exclude `java.lang.Deprecated` during signature 
>> testing (using our original recorded signatures that do include 
>> `java.lang.Deprecated`) but the `java.lang.Deprecated` are still 
>> being verified and complained about. 
>> https://gist.github.com/scottmarlow/e4e7d33088c42ca3433bde98e4ed1d1c 
>> shows the callstack of the invocation to my modified 
>> DefaultExcludeList and some debug prints of passed information.
>>
>> B.  I tried another hack to exclude everything during signature 
>> testing by having DefaultExcludeList#check always throw 
>> `ExcludeException` for all invocations and I still had various 
>> missing classes, added classes, missing super classes, Added 
>> Annotations reported as well 
>> (https://gist.github.com/scottmarlow/9cbcf5703d87f1779a1c73d5ef750d8e 
>> has output from that using the Jakarta EE Signature test tooling that 
>> uses SigTest).
>>
>> Should my second hack (B) that throws `ExcludeException` for all 
>> calls to Except#check, result in there being zero Signature Test 
>> failures?
>>
>> Note that for this testing, we are only passing in the `jakarta` 
>> package names to be checked (via our sig-test-pkg-list_se11.txt 
>> package list file).
>>
>>> Also, below is a list of what I am trying to accomplish (pending 
>>> more discussion + feedback):
>>>
>>> 1.  Do verify that the super class name doesn't change but do not 
>>> verify the super class contents (e.g. no JDK class content checking).
>>> 2.  Do verify that each annotation name is as expected but do not 
>>> verify the actual annotation signature.
>>> 3.  Do verify each public class method/field signature.
>>> 4.  Do fail if any public class method/field are added (for no 
>>> super-setting rule).
>>> 5.  Do fail if any public class method/field are removed (for no 
>>> sub-setting rule).
>>> 6.  To be clear, any JDK class referenced by name in the Jakarta EE 
>>> SPEC API class will still be verified to still reference the same 
>>> class name
>>>
>>> Thanks,
>>>
>>> Scott
>>>
>>>>
>>>> Thanks,
>>>>
>>>> Victor.
>>>>
>>>> On 1/6/21 18:52, Scott Marlow wrote:
>>>>>
>>>>> Fyi, the underlying goal is to allow the Jakarta EE TCK signature 
>>>>> tests to be run against newer Java SE versions than the base Java 
>>>>> SE version used to generate the signatures.
>>>>>
>>>>> Some text from the jakartaee-tck/issues/156 tracking issue [1]:
>>>>> "
>>>>>
>>>>> The recorded signatures include every field and method in every 
>>>>> class and
>>>>> interface in the API, as well as the information for every 
>>>>> superclass or
>>>>> interface. A consequence of this is that it records the signatures of
>>>>> every JDK class used by the API.
>>>>>
>>>>> That means that a different signature file is needed to test an 
>>>>> API on
>>>>> JDK 12 vs JDK 11 (for example), even though the API works 
>>>>> identically on
>>>>> both. There's no way to say "this API requires JDK 11 or newer".
>>>>> "
>>>>>
>>>>> With regard to my previous email below asking about the Deprecated 
>>>>> annotation checking, I am wondering if the two different 
>>>>> Deprecated annotation signatures (Java SE 8 vs SE 11) should be 
>>>>> considered compatible since either form (zero, one or two 
>>>>> parameters) could be executed?  If the answer is that it could 
>>>>> depend on the behaviour expected by the user of the SigTest 
>>>>> tooling, perhaps an extension could be introduced.
>>>>>
>>>>> Scott
>>>>>
>>>>> [1] https://github.com/eclipse-ee4j/jakartaee-tck/issues/156
>>>>>
>>>>> On 1/6/21 1:44 PM, Scott Marlow wrote:
>>>>>>
>>>>>> Hi Victor,
>>>>>>
>>>>>> On 1/5/21 5:33 PM, victor.rudometov at oracle.com wrote:
>>>>>>> Hi Scott,
>>>>>>>
>>>>>>> I would eliminate -exclude java in your setup command making it:
>>>>>>>
>>>>>>> 4.  java -Xmx2g -jar $SIGTEST/sigtestdev.jar Setup -classpath 
>>>>>>> jakarta.persistence-api-3.0.0.jar:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/jre/lib/jce.jar 
>>>>>>> -filename /tmp/jakarta.persistence-api-3.0.0.sig -package jakarta
>>>>>>>
>>>>>>> So the file will store signatures of super classes and 
>>>>>>> interfaces as well (from java* packages).
>>>>>>>
>>>>>>> When running the test -package jakarta option will limit testing 
>>>>>>> to jakarta package and subpackages. It will check members and 
>>>>>>> super classes (so no Map instead of Object is allowed).
>>>>>>
>>>>>>
>>>>>> Thanks, I am trying this now.  It seems to be an improvement.
>>>>>>
>>>>>> However, when generating signatures on Java SE 8 but testing on 
>>>>>> Java SE 11, I cannot seem to ignore/exclude Deprecated annotation 
>>>>>> that changed in Java SE 9.
>>>>>>
>>>>>> The Java SE 8 signature for the jakarta.persistence.Persistence 
>>>>>> class contains:
>>>>>>
>>>>>> "  anno 0 java.lang.Deprecated()
>>>>>> "
>>>>>>
>>>>>> The Java SE 11 signature for the jakarta.persistence.Persistence 
>>>>>> class contains:
>>>>>>
>>>>>> "anno 0 java.lang.Deprecated(boolean forRemoval=false, 
>>>>>> java.lang.String since="")
>>>>>> "
>>>>>>
>>>>>> When running the test command on Java SE 11 with the signature 
>>>>>> generated under Java SE 8 I see:
>>>>>>
>>>>>> "Class jakarta.persistence.Persistence
>>>>>>   Missed Annotations
>>>>>>     PERSISTENCE_PROVIDER:anno 0 java.lang.Deprecated()
>>>>>>     providers:anno 0 java.lang.Deprecated()
>>>>>>   Added Annotations
>>>>>>     PERSISTENCE_PROVIDER:anno 0 java.lang.Deprecated(boolean 
>>>>>> forRemoval=false, java.lang.String since="")
>>>>>>     providers:anno 0 java.lang.Deprecated(boolean 
>>>>>> forRemoval=false, java.lang.String since="")
>>>>>>     finalize():anno 0 java.lang.Deprecated(boolean 
>>>>>> forRemoval=false, java.lang.String since="9")
>>>>>> "
>>>>>>
>>>>>> Is there any SigTest magic built in to ignore the Deprecated 
>>>>>> annotation changes?
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Scott
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>> Victor.
>>>>>>
>>>>>>
>>>>>>
>>>>

More information about the sigtest-dev mailing list