RFR: 7903519 : jtreg/jtharness is missing features for basic crash testing

[andrlos]

On Tue, 12 Nov 2024 09:53:17 GMT, Christian Stein <cstein at openjdk.org> wrote:

>> @sormuras 	feast your eyes on the code below :D
>> 
>> 
>> public class CrashOnlyStatusTransformer implements StatusTransformer {
>>         @Override
>>         public Status transform(Status originalStatus, TestDescription td) {
>>             Status newStatus = originalStatus;
>>             if(originalStatus.getType() == Status.FAILED && ! this.didCrash(td)){
>>                 newStatus = new Status(Status.PASSED, "Just a regular failure.");
>>             }
>>             return newStatus;
>>         }
>> 
>>         private boolean didCrash(TestDescription td){
>>             Pattern pattern = Pattern.compile("^hs_err_pid(\\d+)\.log");
>>             List<String> hs_errs = Arrays.stream(td.getDir().list()).filter(pattern.asPredicate()).collect(Collectors.toList());
>>             return !hs_errs.isEmpty();
>>     }
>> }
>> 
>> 
>> this is an approach that we use for crashtesting with debug jdk builds to separate crashes from regular failures
>
>> [...] this is an approach that we use for crashtesting with debug jdk builds to separate crashes from regular failures
> 
> And sometimes tests do produce `hs_err_pid.log` files and expect/assert them; marking those tests as `PASSED`, right?
> 
> Where (console log, web-view, ...) do you check for such crashes? Manually or with tool/script support?
> 
> Isn't it possible to implement/apply an after-the-fact filter that doesn't rewrite actual run results?

@sormuras it proved to be much harder to filter them after, as the hs_err_pid log is not being copied, plus we have many tests that run via a shell script, so a generic jvm watcher is not an option as would be the option with jvm agent that we use for junit testing.. we need to cover cases, where jvm crashes even tho it has been run via a shell script (or multiple levels of scripts) and watching for hs_err_pid.log proved to be the most reliable and universal approach.

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/235#issuecomment-2470721793