RFR: 7903519 : jtreg/jtharness is missing features for basic crash testing [v5]

[andrlos]

On Thu, 14 Dec 2023 13:16:20 GMT, Vladimir Sitnikov <vsitnikov at openjdk.org> wrote:

>> andrlos has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - missing part of a comment
>>  - addressing more of PR remarks, mostly renaming and StatusTransformer is now initialized with Script instance initialization
>
> src/com/sun/javatest/Script.java line 502:
> 
>> 500:         Status newStatus = statusTransformer.transform(originalStatus, td);
>> 501:         newStatus = new Status(newStatus.getType(), newStatus.getReason() +
>> 502:                     " - The status of this test result has been modified by external code.");
> 
> What do you think of mentioning the transformer name?
> 
> Suggestion:
> 
>                     " - The status of this test result has been modified by " + statusTransfomer.getClass().getName());

might be nice, will add that. Thanks for your suggestion :-)

> src/com/sun/javatest/Script.java line 521:
> 
>> 519:             if (service != null) {
>> 520:                 // Found more than one implementation, throw an exception
>> 521:                 throw new IllegalStateException("Multiple implementations of the StatusTransformer found!");
> 
> An alternative option could be to transform the status in sequence. Then a transformer could ignore the status and let other transformers to decide.

Well yeah I thought about that, but I believe that during the lookup there is no guarantee about the order in which the transformers would come, right? Our usecase would be to use this with a simple transformer that would transform it depending whether a crash happened or not during the execution.. however in some of our tests the crash is a desired outcome and we observe the behavior of our jvm during the crash - whether it links to correct bug reporting channels upon crash etc. In our scenarios we want to mark only crashes as failures and regular failures without crashes as passes.. also passed test in which a crash occurred would be also passed. I believe that there should only be one implementation allowed for now unless someone misses multiple implementations direly in the future, but that person can also squeeze implementation of multiple transformers into a single transformer, right?

I am not purely against, I am just not sure whether the benefits of allowing multiple implementations outweigh the possible errors due to forgotten implementations of transformers.

-------------

PR Review Comment: https://git.openjdk.org/jtharness/pull/57#discussion_r1426707135
PR Review Comment: https://git.openjdk.org/jtharness/pull/57#discussion_r1426717557