RFR: 7903519 : jtreg/jtharness is missing features for basic crash testing

[andrlos]

On Tue, 12 Nov 2024 09:39:03 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> @sormuras 	feast your eyes on the code below :D
>> 
>> 
>> public class CrashOnlyStatusTransformer implements StatusTransformer {
>>         @Override
>>         public Status transform(Status originalStatus, TestDescription td) {
>>             Status newStatus = originalStatus;
>>             if(originalStatus.getType() == Status.FAILED && ! this.didCrash(td)){
>>                 newStatus = new Status(Status.PASSED, "Just a regular failure.");
>>             }
>>             return newStatus;
>>         }
>> 
>>         private boolean didCrash(TestDescription td){
>>             Pattern pattern = Pattern.compile("^hs_err_pid(\\d+)\.log");
>>             List<String> hs_errs = Arrays.stream(td.getDir().list()).filter(pattern.asPredicate()).collect(Collectors.toList());
>>             return !hs_errs.isEmpty();
>>     }
>> }
>> 
>> 
>> this is an approach that we use for crashtesting with debug jdk builds to separate crashes from regular failures
>
> Hello @andrlos, looking at that code you pasted:
> 
> if(originalStatus.getType() == Status.FAILED && ! this.didCrash(td)){
>     newStatus = new Status(Status.PASSED, "Just a regular failure.");
> }
> 
> It looks odd to be marking a failed test as successful. Furthermore, doesn't a crashed JVM result in test status to be `Status.ERROR`?

@jaikiran 
the idea is that if the crashed test is resulting in an error, we still want it to be reported, so those remain unchanged.. if the test has crashed but is reported as failure (usually shell scripts executing JVM) then we want them to stay as they are.. same with the test that test are reported as passed but had a crash occur during execution (we check how the jvm is behaving after crash - e.g. give correct links to report the jvm crash etc..) but when it is reported as failed without the crash happening, we want to ignore those for debug testing... but the code is just a very primitive demo of capabilities of this serviceLookup hook..

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/235#issuecomment-2479038603