From duke at openjdk.org  Fri Nov  1 12:54:17 2024
From: duke at openjdk.org (andrlos)
Date: Fri, 1 Nov 2024 12:54:17 GMT
Subject: RFR: 7903765: wget failed in build.sh in jtreg
Message-ID: <dW16KL1mCfqdUdU2b9QMkChMCV31wK0bDIaB6qNr4zE=.3626b48b-c2d5-4d4c-8081-4b760c6c3722@github.com>

Hi, this is a backport of the original https://github.com/openjdk/jtreg/pull/214 into the 6.1+x branch, since the branch is not currently buildable using make/build.sh script due to reasons in https://bugs.openjdk.org/browse/CODETOOLS-7903765

I also added jcov dependency fix for the same issue as part of the backport.

-------------

Commit messages:
 - 7903765: wget failed in build.sh in jtreg

Changes: https://git.openjdk.org/jtreg/pull/234/files
  Webrev: https://webrevs.openjdk.org/?repo=jtreg&pr=234&range=00
  Issue: https://bugs.openjdk.org/browse/CODETOOLS-7903765
  Stats: 14 lines in 4 files changed: 4 ins; 0 del; 10 mod
  Patch: https://git.openjdk.org/jtreg/pull/234.diff
  Fetch: git fetch https://git.openjdk.org/jtreg.git pull/234/head:pull/234

PR: https://git.openjdk.org/jtreg/pull/234

From jjg at openjdk.org  Fri Nov  1 15:51:42 2024
From: jjg at openjdk.org (Jonathan Gibbons)
Date: Fri, 1 Nov 2024 15:51:42 GMT
Subject: RFR: 7903765: wget failed in build.sh in jtreg
In-Reply-To: <dW16KL1mCfqdUdU2b9QMkChMCV31wK0bDIaB6qNr4zE=.3626b48b-c2d5-4d4c-8081-4b760c6c3722@github.com>
References: <dW16KL1mCfqdUdU2b9QMkChMCV31wK0bDIaB6qNr4zE=.3626b48b-c2d5-4d4c-8081-4b760c6c3722@github.com>
Message-ID: <CxIBdW2gWApKApOvFGzMcYNnDgdpWN34CbC_ZyzPA7Y=.9102ce0b-dc50-4de3-9c44-0b6ec0d88809@github.com>

On Fri, 1 Nov 2024 12:49:58 GMT, andrlos <duke at openjdk.org> wrote:

> Hi, this is a backport of the original https://github.com/openjdk/jtreg/pull/214 into the 6.1+x branch, since the branch is not currently buildable using make/build.sh script due to reasons in https://bugs.openjdk.org/browse/CODETOOLS-7903765
> 
> I also added jcov dependency fix for the same issue as part of the backport.

See various bot suggestions, including the backport suggestion

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/234#issuecomment-2452106359

From duke at openjdk.org  Mon Nov  4 10:37:40 2024
From: duke at openjdk.org (andrlos)
Date: Mon, 4 Nov 2024 10:37:40 GMT
Subject: RFR: Backport 593d6c46f52d06a227a4b26b392b7ebd6cc8a5e7
In-Reply-To: <CxIBdW2gWApKApOvFGzMcYNnDgdpWN34CbC_ZyzPA7Y=.9102ce0b-dc50-4de3-9c44-0b6ec0d88809@github.com>
References: <dW16KL1mCfqdUdU2b9QMkChMCV31wK0bDIaB6qNr4zE=.3626b48b-c2d5-4d4c-8081-4b760c6c3722@github.com>
 <CxIBdW2gWApKApOvFGzMcYNnDgdpWN34CbC_ZyzPA7Y=.9102ce0b-dc50-4de3-9c44-0b6ec0d88809@github.com>
Message-ID: <QAw0kapxu4VuWwCX88j9_M10HQenjmv1sUL9FwUHv3c=.ca126436-c6ee-4dfd-ab08-f5c448c29b49@github.com>

On Fri, 1 Nov 2024 15:49:03 GMT, Jonathan Gibbons <jjg at openjdk.org> wrote:

>> Hi, this is a backport of the original https://github.com/openjdk/jtreg/pull/214 into the 6.1+x branch, since the branch is not currently buildable using make/build.sh script due to reasons in https://bugs.openjdk.org/browse/CODETOOLS-7903765
>> 
>> I also added jcov dependency fix for the same issue as part of the backport.
>
> See various bot suggestions, including the backport suggestion

@jonathan-gibbons I am very sorry I missed that. Should be better now.

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/234#issuecomment-2454355138

From cstein at openjdk.org  Mon Nov  4 15:31:42 2024
From: cstein at openjdk.org (Christian Stein)
Date: Mon, 4 Nov 2024 15:31:42 GMT
Subject: RFR: Backport 593d6c46f52d06a227a4b26b392b7ebd6cc8a5e7
In-Reply-To: <dW16KL1mCfqdUdU2b9QMkChMCV31wK0bDIaB6qNr4zE=.3626b48b-c2d5-4d4c-8081-4b760c6c3722@github.com>
References: <dW16KL1mCfqdUdU2b9QMkChMCV31wK0bDIaB6qNr4zE=.3626b48b-c2d5-4d4c-8081-4b760c6c3722@github.com>
Message-ID: <SZNsChEoZHaB_z8Wy2QhtFGz_jgwAEuOHz_b72jc0to=.6dcfb4aa-f949-433b-a9ed-25dacef6dfcc@github.com>

On Fri, 1 Nov 2024 12:49:58 GMT, andrlos <duke at openjdk.org> wrote:

> Hi, this is a backport of the original https://github.com/openjdk/jtreg/pull/214 into the 6.1+x branch, since the branch is not currently buildable using make/build.sh script due to reasons in https://bugs.openjdk.org/browse/CODETOOLS-7903765
> 
> I also added jcov dependency fix for the same issue as part of the backport.

In order for Skara to workout this backport, shouldn't the process be started with a `/backport ...` command comment in the original commit? That original commit for "7903765: wget failed in build.sh in jtreg" is: https://github.com/openjdk/jtreg/commit/ca8c1ecf924c4ea40fc13a907559aaa90f3d0196 -- why did you use https://github.com/openjdk/jtreg/commit/593d6c46f52d06a227a4b26b392b7ebd6cc8a5e7 here? Merging two commits in a single backport is not supported by Skara, I guess.

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/234#issuecomment-2455015942

From duke at openjdk.org  Wed Nov  6 01:49:01 2024
From: duke at openjdk.org (andrlos)
Date: Wed, 6 Nov 2024 01:49:01 GMT
Subject: RFR: 7903519 : jtreg/jtharness is missing features for basic crash
 testing
Message-ID: <1k_V7Eg7B_QW6-WNSNhWWezgOgfLFLnS5zK85TfTiZ0=.b616aac4-079f-4023-adb6-7e4542c7d856@github.com>

provides SPI for enabling external status transformations of failed tests

this is a continuation of efforts after https://github.com/openjdk/jtharness/pull/59 

Requires newest jtharness build (not even tagged yet) that includes above mentioned change to be compiled succesfully

The main idea is to provide a unified StatusTransformer interface, that can be externally implemented by users and added to a classpath in a separate jar to allow modifications of test execution status based on some elementary analysis. This can be easily used for crashtesting (filtering out only tests with jvm crashes).

-------------

Commit messages:
 - SPI for enabling external status transformations of failed tests

Changes: https://git.openjdk.org/jtreg/pull/235/files
  Webrev: https://webrevs.openjdk.org/?repo=jtreg&pr=235&range=00
  Issue: https://bugs.openjdk.org/browse/CODETOOLS-7903519
  Stats: 94 lines in 2 files changed: 94 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jtreg/pull/235.diff
  Fetch: git fetch https://git.openjdk.org/jtreg.git pull/235/head:pull/235

PR: https://git.openjdk.org/jtreg/pull/235

From cstein at openjdk.org  Wed Nov  6 12:39:43 2024
From: cstein at openjdk.org (Christian Stein)
Date: Wed, 6 Nov 2024 12:39:43 GMT
Subject: RFR: 7903519 : jtreg/jtharness is missing features for basic crash
 testing
In-Reply-To: <1k_V7Eg7B_QW6-WNSNhWWezgOgfLFLnS5zK85TfTiZ0=.b616aac4-079f-4023-adb6-7e4542c7d856@github.com>
References: <1k_V7Eg7B_QW6-WNSNhWWezgOgfLFLnS5zK85TfTiZ0=.b616aac4-079f-4023-adb6-7e4542c7d856@github.com>
Message-ID: <uA9y4ZKCN_-Rsri91_WEPdahbaZOPe6W-fn5YzhP7QA=.d1764c3f-eab1-4689-b0e5-22d0ead28a18@github.com>

On Wed, 6 Nov 2024 01:45:22 GMT, andrlos <duke at openjdk.org> wrote:

> provides SPI for enabling external status transformations of failed tests
> 
> this is a continuation of efforts after https://github.com/openjdk/jtharness/pull/59 
> 
> Requires newest jtharness build (not even tagged yet) that includes above mentioned change to be compiled succesfully
> 
> The main idea is to provide a unified StatusTransformer interface, that can be externally implemented by users and added to a classpath in a separate jar to allow modifications of test execution status based on some elementary analysis. This can be easily used for crashtesting (filtering out only tests with jvm crashes).

A test status listener (in the sense of JUnit's [TestWatcher](https://junit.org/junit5/docs/current/user-guide/#extensions-test-result-processing)) would be okay-ish, but a general test status transforming SPI is too intrusive and intransparent ... and also exposing an internal field of `jtharness`'s `Script` class.

In the light of the above, I tend to close this PR.

What would an implementation handling "crashtesting" look like? Do you have a draft for it?

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/235#issuecomment-2459642948

From duke at openjdk.org  Mon Nov 11 18:22:08 2024
From: duke at openjdk.org (andrlos)
Date: Mon, 11 Nov 2024 18:22:08 GMT
Subject: RFR: 7903519 : jtreg/jtharness is missing features for basic crash
 testing
In-Reply-To: <uA9y4ZKCN_-Rsri91_WEPdahbaZOPe6W-fn5YzhP7QA=.d1764c3f-eab1-4689-b0e5-22d0ead28a18@github.com>
References: <1k_V7Eg7B_QW6-WNSNhWWezgOgfLFLnS5zK85TfTiZ0=.b616aac4-079f-4023-adb6-7e4542c7d856@github.com>
 <uA9y4ZKCN_-Rsri91_WEPdahbaZOPe6W-fn5YzhP7QA=.d1764c3f-eab1-4689-b0e5-22d0ead28a18@github.com>
Message-ID: <8Vqx8LlIfN9WFhOFOpF7v1dAul4balD5EdGktzip9hA=.f6a673f6-0089-41ec-8241-8cc5f01bcfb5@github.com>

On Wed, 6 Nov 2024 12:37:15 GMT, Christian Stein <cstein at openjdk.org> wrote:

>> provides SPI for enabling external status transformations of failed tests
>> 
>> this is a continuation of efforts after https://github.com/openjdk/jtharness/pull/59 
>> 
>> Requires newest jtharness build (not even tagged yet) that includes above mentioned change to be compiled succesfully
>> 
>> The main idea is to provide a unified StatusTransformer interface, that can be externally implemented by users and added to a classpath in a separate jar to allow modifications of test execution status based on some elementary analysis. This can be easily used for crashtesting (filtering out only tests with jvm crashes).
>
> A test status listener (in the sense of JUnit's [TestWatcher](https://junit.org/junit5/docs/current/user-guide/#extensions-test-result-processing)) would be okay-ish, but a general test status transforming SPI is too intrusive and intransparent ... and also exposing an internal field of `jtharness`'s `Script` class.
> 
> In the light of the above, I tend to close this PR.
> 
> What would an implementation handling "crashtesting" look like? Do you have a draft for it?

@sormuras 	feast your eyes on the code below :D


public class CrashOnlyStatusTransformer implements StatusTransformer {
        @Override
        public Status transform(Status originalStatus, TestDescription td) {
            Status newStatus = originalStatus;
            if(originalStatus.getType() == Status.FAILED && ! this.didCrash(td)){
                newStatus = new Status(Status.PASSED, "Just a regular failure.");
            }
            return newStatus;
        }

        private boolean didCrash(TestDescription td){
            Pattern pattern = Pattern.compile("^hs_err_pid(\\d+)\.log");
            List<String> hs_errs = Arrays.stream(td.getDir().list()).filter(pattern.asPredicate()).collect(Collectors.toList());
            return !hs_errs.isEmpty();
    }
}


this is an approach that we use for crashtesting with debug jdk builds to separate crashes from regular failures

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/235#issuecomment-2468779383

From jpai at openjdk.org  Tue Nov 12 09:42:17 2024
From: jpai at openjdk.org (Jaikiran Pai)
Date: Tue, 12 Nov 2024 09:42:17 GMT
Subject: RFR: 7903519 : jtreg/jtharness is missing features for basic crash
 testing
In-Reply-To: <8Vqx8LlIfN9WFhOFOpF7v1dAul4balD5EdGktzip9hA=.f6a673f6-0089-41ec-8241-8cc5f01bcfb5@github.com>
References: <1k_V7Eg7B_QW6-WNSNhWWezgOgfLFLnS5zK85TfTiZ0=.b616aac4-079f-4023-adb6-7e4542c7d856@github.com>
 <uA9y4ZKCN_-Rsri91_WEPdahbaZOPe6W-fn5YzhP7QA=.d1764c3f-eab1-4689-b0e5-22d0ead28a18@github.com>
 <8Vqx8LlIfN9WFhOFOpF7v1dAul4balD5EdGktzip9hA=.f6a673f6-0089-41ec-8241-8cc5f01bcfb5@github.com>
Message-ID: <ts-j8xJf1nIuxc7TPq43byL3C3qOKc4NA3SqrhUPasY=.0b915b11-4660-49ce-bd58-33d16a3d3efa@github.com>

On Mon, 11 Nov 2024 18:19:48 GMT, andrlos <duke at openjdk.org> wrote:

>> A test status listener (in the sense of JUnit's [TestWatcher](https://junit.org/junit5/docs/current/user-guide/#extensions-test-result-processing)) would be okay-ish, but a general test status transforming SPI is too intrusive and intransparent ... and also exposing an internal field of `jtharness`'s `Script` class.
>> 
>> In the light of the above, I tend to close this PR.
>> 
>> What would an implementation handling "crashtesting" look like? Do you have a draft for it?
>
> @sormuras 	feast your eyes on the code below :D
> 
> 
> public class CrashOnlyStatusTransformer implements StatusTransformer {
>         @Override
>         public Status transform(Status originalStatus, TestDescription td) {
>             Status newStatus = originalStatus;
>             if(originalStatus.getType() == Status.FAILED && ! this.didCrash(td)){
>                 newStatus = new Status(Status.PASSED, "Just a regular failure.");
>             }
>             return newStatus;
>         }
> 
>         private boolean didCrash(TestDescription td){
>             Pattern pattern = Pattern.compile("^hs_err_pid(\\d+)\.log");
>             List<String> hs_errs = Arrays.stream(td.getDir().list()).filter(pattern.asPredicate()).collect(Collectors.toList());
>             return !hs_errs.isEmpty();
>     }
> }
> 
> 
> this is an approach that we use for crashtesting with debug jdk builds to separate crashes from regular failures

Hello @andrlos, looking at that code you pasted:

if(originalStatus.getType() == Status.FAILED && ! this.didCrash(td)){
    newStatus = new Status(Status.PASSED, "Just a regular failure.");
}

It looks odd to be marking a failed test as successful. Furthermore, doesn't a crashed JVM result in test status to be `Status.ERROR`?

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/235#issuecomment-2470046873

From cstein at openjdk.org  Tue Nov 12 09:57:13 2024
From: cstein at openjdk.org (Christian Stein)
Date: Tue, 12 Nov 2024 09:57:13 GMT
Subject: RFR: 7903519 : jtreg/jtharness is missing features for basic crash
 testing
In-Reply-To: <8Vqx8LlIfN9WFhOFOpF7v1dAul4balD5EdGktzip9hA=.f6a673f6-0089-41ec-8241-8cc5f01bcfb5@github.com>
References: <1k_V7Eg7B_QW6-WNSNhWWezgOgfLFLnS5zK85TfTiZ0=.b616aac4-079f-4023-adb6-7e4542c7d856@github.com>
 <uA9y4ZKCN_-Rsri91_WEPdahbaZOPe6W-fn5YzhP7QA=.d1764c3f-eab1-4689-b0e5-22d0ead28a18@github.com>
 <8Vqx8LlIfN9WFhOFOpF7v1dAul4balD5EdGktzip9hA=.f6a673f6-0089-41ec-8241-8cc5f01bcfb5@github.com>
Message-ID: <n9SzF-5HeuT0XXmYS8lIxl3he5aZBRlM1Tl5b0fYI-M=.af90f2ed-5784-49bf-9faa-d26dbcfaf747@github.com>

On Mon, 11 Nov 2024 18:19:48 GMT, andrlos <duke at openjdk.org> wrote:

> [...] this is an approach that we use for crashtesting with debug jdk builds to separate crashes from regular failures

And sometimes tests do produce `hs_err_pid.log` files and expect/assert them; marking those tests as `PASSED`, right?

Where (console log, web-view, ...) do you check for such crashes? Manually or with tool/script support?

Isn't it possible to implement/apply an after-the-fact filter that doesn't rewrite actual run results?

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/235#issuecomment-2470078735

From duke at openjdk.org  Tue Nov 12 14:45:17 2024
From: duke at openjdk.org (andrlos)
Date: Tue, 12 Nov 2024 14:45:17 GMT
Subject: RFR: 7903519 : jtreg/jtharness is missing features for basic crash
 testing
In-Reply-To: <n9SzF-5HeuT0XXmYS8lIxl3he5aZBRlM1Tl5b0fYI-M=.af90f2ed-5784-49bf-9faa-d26dbcfaf747@github.com>
References: <1k_V7Eg7B_QW6-WNSNhWWezgOgfLFLnS5zK85TfTiZ0=.b616aac4-079f-4023-adb6-7e4542c7d856@github.com>
 <uA9y4ZKCN_-Rsri91_WEPdahbaZOPe6W-fn5YzhP7QA=.d1764c3f-eab1-4689-b0e5-22d0ead28a18@github.com>
 <8Vqx8LlIfN9WFhOFOpF7v1dAul4balD5EdGktzip9hA=.f6a673f6-0089-41ec-8241-8cc5f01bcfb5@github.com>
 <n9SzF-5HeuT0XXmYS8lIxl3he5aZBRlM1Tl5b0fYI-M=.af90f2ed-5784-49bf-9faa-d26dbcfaf747@github.com>
Message-ID: <UV8lDedyBwiX5L8N14-Ndx5xkCzYg0CKY5V8gBVFhY4=.65d37db2-cd44-4ca3-b96d-879132a3cbea@github.com>

On Tue, 12 Nov 2024 09:53:17 GMT, Christian Stein <cstein at openjdk.org> wrote:

>> @sormuras 	feast your eyes on the code below :D
>> 
>> 
>> public class CrashOnlyStatusTransformer implements StatusTransformer {
>>         @Override
>>         public Status transform(Status originalStatus, TestDescription td) {
>>             Status newStatus = originalStatus;
>>             if(originalStatus.getType() == Status.FAILED && ! this.didCrash(td)){
>>                 newStatus = new Status(Status.PASSED, "Just a regular failure.");
>>             }
>>             return newStatus;
>>         }
>> 
>>         private boolean didCrash(TestDescription td){
>>             Pattern pattern = Pattern.compile("^hs_err_pid(\\d+)\.log");
>>             List<String> hs_errs = Arrays.stream(td.getDir().list()).filter(pattern.asPredicate()).collect(Collectors.toList());
>>             return !hs_errs.isEmpty();
>>     }
>> }
>> 
>> 
>> this is an approach that we use for crashtesting with debug jdk builds to separate crashes from regular failures
>
>> [...] this is an approach that we use for crashtesting with debug jdk builds to separate crashes from regular failures
> 
> And sometimes tests do produce `hs_err_pid.log` files and expect/assert them; marking those tests as `PASSED`, right?
> 
> Where (console log, web-view, ...) do you check for such crashes? Manually or with tool/script support?
> 
> Isn't it possible to implement/apply an after-the-fact filter that doesn't rewrite actual run results?

@sormuras it proved to be much harder to filter them after, as the hs_err_pid log is not being copied, plus we have many tests that run via a shell script, so a generic jvm watcher is not an option as would be the option with jvm agent that we use for junit testing.. we need to cover cases, where jvm crashes even tho it has been run via a shell script (or multiple levels of scripts) and watching for hs_err_pid.log proved to be the most reliable and universal approach.

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/235#issuecomment-2470721793

From duke at openjdk.org  Tue Nov 12 15:09:25 2024
From: duke at openjdk.org (andrlos)
Date: Tue, 12 Nov 2024 15:09:25 GMT
Subject: RFR: 7903765: wget failed in build.sh in jtreg
In-Reply-To: <SZNsChEoZHaB_z8Wy2QhtFGz_jgwAEuOHz_b72jc0to=.6dcfb4aa-f949-433b-a9ed-25dacef6dfcc@github.com>
References: <dW16KL1mCfqdUdU2b9QMkChMCV31wK0bDIaB6qNr4zE=.3626b48b-c2d5-4d4c-8081-4b760c6c3722@github.com>
 <SZNsChEoZHaB_z8Wy2QhtFGz_jgwAEuOHz_b72jc0to=.6dcfb4aa-f949-433b-a9ed-25dacef6dfcc@github.com>
Message-ID: <IvvE5uKMTFaoiaK3CVZnLwqMwxsdZfIJdUR8QmkWupA=.5abebb64-3e53-4e4d-848c-e3a97f628e72@github.com>

On Mon, 4 Nov 2024 15:28:48 GMT, Christian Stein <cstein at openjdk.org> wrote:

>> Hi, this is a backport of the original https://github.com/openjdk/jtreg/pull/214 into the 6.1+x branch, since the branch is not currently buildable using make/build.sh script due to reasons in https://bugs.openjdk.org/browse/CODETOOLS-7903765
>> 
>> I also added jcov dependency fix for the same issue as part of the backport.
>
> In order for Skara to workout this backport, shouldn't the process be started with a `/backport ...` command comment in the original commit? That original commit for "7903765: wget failed in build.sh in jtreg" is: https://github.com/openjdk/jtreg/commit/ca8c1ecf924c4ea40fc13a907559aaa90f3d0196 -- why did you use https://github.com/openjdk/jtreg/commit/593d6c46f52d06a227a4b26b392b7ebd6cc8a5e7 here? Merging two commits in a single backport is not supported by Skara, I guess.
> 
> Find more details at https://wiki.openjdk.org/display/SKARA/Backports

@sormuras, I have honestly no idea where I got that hash at the time.. might have been late and I copypasted it from a wrong terminal or somethin.. I am sorry for my mistake.

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/234#issuecomment-2470779560

From duke at openjdk.org  Tue Nov 12 15:09:25 2024
From: duke at openjdk.org (duke)
Date: Tue, 12 Nov 2024 15:09:25 GMT
Subject: RFR: 7903765: wget failed in build.sh in jtreg
In-Reply-To: <dW16KL1mCfqdUdU2b9QMkChMCV31wK0bDIaB6qNr4zE=.3626b48b-c2d5-4d4c-8081-4b760c6c3722@github.com>
References: <dW16KL1mCfqdUdU2b9QMkChMCV31wK0bDIaB6qNr4zE=.3626b48b-c2d5-4d4c-8081-4b760c6c3722@github.com>
Message-ID: <uZwzivPWQDSgmeWt9Np6IuitWdmrk6SneAYEIv5CD8g=.10eaa3bb-f602-424a-8229-719e0ca5ed1c@github.com>

On Fri, 1 Nov 2024 12:49:58 GMT, andrlos <duke at openjdk.org> wrote:

> Hi, this is a backport of the original https://github.com/openjdk/jtreg/pull/214 into the 6.1+x branch, since the branch is not currently buildable using make/build.sh script due to reasons in https://bugs.openjdk.org/browse/CODETOOLS-7903765
> 
> I also added jcov dependency fix for the same issue as part of the backport.

@andrlos 
Your change (at version a267df1eb4e09e9c89c9ab6a33d6a41c32d43712) is now ready to be sponsored by a Committer.

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/234#issuecomment-2470780364

From duke at openjdk.org  Tue Nov 12 15:09:25 2024
From: duke at openjdk.org (andrlos)
Date: Tue, 12 Nov 2024 15:09:25 GMT
Subject: RFR: 7903765: wget failed in build.sh in jtreg
In-Reply-To: <dW16KL1mCfqdUdU2b9QMkChMCV31wK0bDIaB6qNr4zE=.3626b48b-c2d5-4d4c-8081-4b760c6c3722@github.com>
References: <dW16KL1mCfqdUdU2b9QMkChMCV31wK0bDIaB6qNr4zE=.3626b48b-c2d5-4d4c-8081-4b760c6c3722@github.com>
Message-ID: <zruqaBl_9yT47W6L7b7ZuDDlaG5k61i2NNY4XOvbOF4=.74241eca-c951-40c9-8b30-3b6d0b56bf5f@github.com>

On Fri, 1 Nov 2024 12:49:58 GMT, andrlos <duke at openjdk.org> wrote:

> Hi, this is a backport of the original https://github.com/openjdk/jtreg/pull/214 into the 6.1+x branch, since the branch is not currently buildable using make/build.sh script due to reasons in https://bugs.openjdk.org/browse/CODETOOLS-7903765
> 
> I also added jcov dependency fix for the same issue as part of the backport.

if someone would be willing to sponsor this now? I do not have commiter priviledges here.

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/234#issuecomment-2470782283

From duke at openjdk.org  Tue Nov 12 15:59:22 2024
From: duke at openjdk.org (andrlos)
Date: Tue, 12 Nov 2024 15:59:22 GMT
Subject: Integrated: 7903765: wget failed in build.sh in jtreg
In-Reply-To: <dW16KL1mCfqdUdU2b9QMkChMCV31wK0bDIaB6qNr4zE=.3626b48b-c2d5-4d4c-8081-4b760c6c3722@github.com>
References: <dW16KL1mCfqdUdU2b9QMkChMCV31wK0bDIaB6qNr4zE=.3626b48b-c2d5-4d4c-8081-4b760c6c3722@github.com>
Message-ID: <yOFqSJCJZOY1EaNCMlmRLw3lBO_NYxUCh1skoMJHg5Q=.af5ea78c-07e1-481f-9a31-eb20f198d8f9@github.com>

On Fri, 1 Nov 2024 12:49:58 GMT, andrlos <duke at openjdk.org> wrote:

> Hi, this is a backport of the original https://github.com/openjdk/jtreg/pull/214 into the 6.1+x branch, since the branch is not currently buildable using make/build.sh script due to reasons in https://bugs.openjdk.org/browse/CODETOOLS-7903765
> 
> I also added jcov dependency fix for the same issue as part of the backport.

This pull request has now been integrated.

Changeset: 294ce427
Author:    Jaikiran Pai <jpai at openjdk.org>
Committer: Christian Stein <cstein at openjdk.org>
URL:       https://git.openjdk.org/jtreg/commit/294ce4278ef76832cee68444f175aa914651de1b
Stats:     14 lines in 4 files changed: 4 ins; 0 del; 10 mod

7903765: wget failed in build.sh in jtreg

Backport-of: ca8c1ecf924c4ea40fc13a907559aaa90f3d0196

-------------

PR: https://git.openjdk.org/jtreg/pull/234

From lujaniuk at openjdk.org  Wed Nov 13 13:20:50 2024
From: lujaniuk at openjdk.org (Ludvig Janiuk)
Date: Wed, 13 Nov 2024 13:20:50 GMT
Subject: RFR: 7903883: --verify-exclude test existence and format checks are
 broken
Message-ID: <GTvesx96d3nXWjxlzTX9PxV2UdfLyRgOFNmgJNRlaj8=.59aa2e76-b10c-4182-a146-27e2881af01c@github.com>

The way this feature selected "the full set of tests" was actually getting "the set of tests to be run". But often, we run some subset through either specifying test files, folders, or groups. Our problemlist files will still have entries for other tests we were not running right now.

As a solution, --verify-exclude needs to compute the maximal set of tests that could be run for all test suites that any of the tests being run are a part of. I've implemented this by creating a dummy testmanager and passing no parameters to it.

Additionally, the regex for problemlist format had a typo.

-------------

Commit messages:
 - ws
 - ws
 - group invoke test
 - cleanup2
 - cleanup
 - CODETOOLS-7903883 --verify-exclude test existence and format checks are broken
 - failing test

Changes: https://git.openjdk.org/jtreg/pull/236/files
  Webrev: https://webrevs.openjdk.org/?repo=jtreg&pr=236&range=00
  Issue: https://bugs.openjdk.org/browse/CODETOOLS-7903883
  Stats: 83 lines in 6 files changed: 71 ins; 1 del; 11 mod
  Patch: https://git.openjdk.org/jtreg/pull/236.diff
  Fetch: git fetch https://git.openjdk.org/jtreg.git pull/236/head:pull/236

PR: https://git.openjdk.org/jtreg/pull/236

From cstein at openjdk.org  Wed Nov 13 15:58:18 2024
From: cstein at openjdk.org (Christian Stein)
Date: Wed, 13 Nov 2024 15:58:18 GMT
Subject: RFR: 7903883: --verify-exclude test existence and format checks
 are broken
In-Reply-To: <GTvesx96d3nXWjxlzTX9PxV2UdfLyRgOFNmgJNRlaj8=.59aa2e76-b10c-4182-a146-27e2881af01c@github.com>
References: <GTvesx96d3nXWjxlzTX9PxV2UdfLyRgOFNmgJNRlaj8=.59aa2e76-b10c-4182-a146-27e2881af01c@github.com>
Message-ID: <pgN11L6sCieuei_nZTL2pPj89Tt0-FvE49Y89q5MAcA=.1656ad56-4631-457d-830b-5e686b325fad@github.com>

On Wed, 13 Nov 2024 09:11:06 GMT, Ludvig Janiuk <lujaniuk at openjdk.org> wrote:

> The way this feature selected "the full set of tests" was actually getting "the set of tests to be run". But often, we run some subset through either specifying test files, folders, or groups. Our problemlist files will still have entries for other tests we were not running right now.
> 
> As a solution, --verify-exclude needs to compute the maximal set of tests that could be run for all test suites that any of the tests being run are a part of. I've implemented this by creating a dummy testmanager and passing no parameters to it.
> 
> Additionally, the regex for problemlist format had a typo.

Looks good so far, left some requests regarding a `TODO` comment and optional verbose output.

src/share/classes/com/sun/javatest/regtest/tool/Tool.java line 1184:

> 1182: 
> 1183:         // TODO We know set of suites here, could save/do dummy
> 1184: 

Suggestion:



Please don't add `TODO` comments ? just do it, or don't.

src/share/classes/com/sun/javatest/regtest/tool/Tool.java line 1429:

> 1427:             for (Iterator<TestResult> iter = getResultsIterator(params); iter.hasNext(); ) {
> 1428:                 TestResult tr = iter.next();
> 1429:                 out.println(tr.getTestName());

This print statement would now produce many output lines by default - perhaps, re-add and guard it via the `--verbose` flag?

src/share/classes/com/sun/javatest/regtest/tool/Tool.java line 1436:

> 1434:         List<String> validTestNames = new ArrayList<String>();
> 1435:         for (RegressionTestSuite ts: dummyTestManager.getTestSuites()) {
> 1436:             out.println(i18n.getString("main.tests.suite", ts.getRootDir()));

Guard this output line by the `--verbose` option?

-------------

Changes requested by cstein (Committer).

PR Review: https://git.openjdk.org/jtreg/pull/236#pullrequestreview-2433657556
PR Review Comment: https://git.openjdk.org/jtreg/pull/236#discussion_r1840650394
PR Review Comment: https://git.openjdk.org/jtreg/pull/236#discussion_r1840668862
PR Review Comment: https://git.openjdk.org/jtreg/pull/236#discussion_r1840658364

From lujaniuk at openjdk.org  Fri Nov 15 09:20:39 2024
From: lujaniuk at openjdk.org (Ludvig Janiuk)
Date: Fri, 15 Nov 2024 09:20:39 GMT
Subject: RFR: 7903883: --verify-exclude test existence and format checks
 are broken [v2]
In-Reply-To: <GTvesx96d3nXWjxlzTX9PxV2UdfLyRgOFNmgJNRlaj8=.59aa2e76-b10c-4182-a146-27e2881af01c@github.com>
References: <GTvesx96d3nXWjxlzTX9PxV2UdfLyRgOFNmgJNRlaj8=.59aa2e76-b10c-4182-a146-27e2881af01c@github.com>
Message-ID: <1ju9rkwe3zB5-d2gv4lGv4ZhwCQSzzGjEbPO9TQpeoU=.0bb33e71-1729-42d8-81b0-bc4a4e7b678a@github.com>

> The way this feature selected "the full set of tests" was actually getting "the set of tests to be run". But often, we run some subset through either specifying test files, folders, or groups. Our problemlist files will still have entries for other tests we were not running right now.
> 
> As a solution, --verify-exclude needs to compute the maximal set of tests that could be run for all test suites that any of the tests being run are a part of. I've implemented this by creating a dummy testmanager and passing no parameters to it.
> 
> Additionally, the regex for problemlist format had a typo.

Ludvig Janiuk has updated the pull request incrementally with two additional commits since the last revision:

 - verbose output
 - rm todo

-------------

Changes:
  - all: https://git.openjdk.org/jtreg/pull/236/files
  - new: https://git.openjdk.org/jtreg/pull/236/files/65869e8b..fd5e203f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jtreg&pr=236&range=01
 - incr: https://webrevs.openjdk.org/?repo=jtreg&pr=236&range=00-01

  Stats: 54 lines in 3 files changed: 50 ins; 2 del; 2 mod
  Patch: https://git.openjdk.org/jtreg/pull/236.diff
  Fetch: git fetch https://git.openjdk.org/jtreg.git pull/236/head:pull/236

PR: https://git.openjdk.org/jtreg/pull/236

From lujaniuk at openjdk.org  Fri Nov 15 09:20:40 2024
From: lujaniuk at openjdk.org (Ludvig Janiuk)
Date: Fri, 15 Nov 2024 09:20:40 GMT
Subject: RFR: 7903883: --verify-exclude test existence and format checks
 are broken [v2]
In-Reply-To: <pgN11L6sCieuei_nZTL2pPj89Tt0-FvE49Y89q5MAcA=.1656ad56-4631-457d-830b-5e686b325fad@github.com>
References: <GTvesx96d3nXWjxlzTX9PxV2UdfLyRgOFNmgJNRlaj8=.59aa2e76-b10c-4182-a146-27e2881af01c@github.com>
 <pgN11L6sCieuei_nZTL2pPj89Tt0-FvE49Y89q5MAcA=.1656ad56-4631-457d-830b-5e686b325fad@github.com>
Message-ID: <FSRXIJhjeeVYqg01H2CE8I7Z5aY7w43FaPoWDjylx_g=.184719d3-a0b2-4e62-afa4-4af159d8814e@github.com>

On Wed, 13 Nov 2024 15:54:04 GMT, Christian Stein <cstein at openjdk.org> wrote:

>> Ludvig Janiuk has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - verbose output
>>  - rm todo
>
> Looks good so far, left some requests regarding a `TODO` comment and optional verbose output.

Thank you for reviewing @sormuras. I agree with all your comments and have addressed them. I chose to put the test names behind -va because that's a huuuuge list for the jdk.

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/236#issuecomment-2478312303

From duke at openjdk.org  Fri Nov 15 14:43:08 2024
From: duke at openjdk.org (andrlos)
Date: Fri, 15 Nov 2024 14:43:08 GMT
Subject: RFR: 7903519 : jtreg/jtharness is missing features for basic crash
 testing
In-Reply-To: <ts-j8xJf1nIuxc7TPq43byL3C3qOKc4NA3SqrhUPasY=.0b915b11-4660-49ce-bd58-33d16a3d3efa@github.com>
References: <1k_V7Eg7B_QW6-WNSNhWWezgOgfLFLnS5zK85TfTiZ0=.b616aac4-079f-4023-adb6-7e4542c7d856@github.com>
 <uA9y4ZKCN_-Rsri91_WEPdahbaZOPe6W-fn5YzhP7QA=.d1764c3f-eab1-4689-b0e5-22d0ead28a18@github.com>
 <8Vqx8LlIfN9WFhOFOpF7v1dAul4balD5EdGktzip9hA=.f6a673f6-0089-41ec-8241-8cc5f01bcfb5@github.com>
 <ts-j8xJf1nIuxc7TPq43byL3C3qOKc4NA3SqrhUPasY=.0b915b11-4660-49ce-bd58-33d16a3d3efa@github.com>
Message-ID: <tJ6aTK2IiW3-JB7PSQhm0UUv75q_-wXjLfwkriEX-I8=.45f30c00-3bac-40c7-90b1-00512aa10873@github.com>

On Tue, 12 Nov 2024 09:39:03 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> @sormuras 	feast your eyes on the code below :D
>> 
>> 
>> public class CrashOnlyStatusTransformer implements StatusTransformer {
>>         @Override
>>         public Status transform(Status originalStatus, TestDescription td) {
>>             Status newStatus = originalStatus;
>>             if(originalStatus.getType() == Status.FAILED && ! this.didCrash(td)){
>>                 newStatus = new Status(Status.PASSED, "Just a regular failure.");
>>             }
>>             return newStatus;
>>         }
>> 
>>         private boolean didCrash(TestDescription td){
>>             Pattern pattern = Pattern.compile("^hs_err_pid(\\d+)\.log");
>>             List<String> hs_errs = Arrays.stream(td.getDir().list()).filter(pattern.asPredicate()).collect(Collectors.toList());
>>             return !hs_errs.isEmpty();
>>     }
>> }
>> 
>> 
>> this is an approach that we use for crashtesting with debug jdk builds to separate crashes from regular failures
>
> Hello @andrlos, looking at that code you pasted:
> 
> if(originalStatus.getType() == Status.FAILED && ! this.didCrash(td)){
>     newStatus = new Status(Status.PASSED, "Just a regular failure.");
> }
> 
> It looks odd to be marking a failed test as successful. Furthermore, doesn't a crashed JVM result in test status to be `Status.ERROR`?

@jaikiran 
the idea is that if the crashed test is resulting in an error, we still want it to be reported, so those remain unchanged.. if the test has crashed but is reported as failure (usually shell scripts executing JVM) then we want them to stay as they are.. same with the test that test are reported as passed but had a crash occur during execution (we check how the jvm is behaving after crash - e.g. give correct links to report the jvm crash etc..) but when it is reported as failed without the crash happening, we want to ignore those for debug testing... but the code is just a very primitive demo of capabilities of this serviceLookup hook..

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/235#issuecomment-2479038603

From cstein at openjdk.org  Wed Nov 20 10:34:34 2024
From: cstein at openjdk.org (Christian Stein)
Date: Wed, 20 Nov 2024 10:34:34 GMT
Subject: RFR: 7903883: --verify-exclude test existence and format checks
 are broken [v2]
In-Reply-To: <1ju9rkwe3zB5-d2gv4lGv4ZhwCQSzzGjEbPO9TQpeoU=.0bb33e71-1729-42d8-81b0-bc4a4e7b678a@github.com>
References: <GTvesx96d3nXWjxlzTX9PxV2UdfLyRgOFNmgJNRlaj8=.59aa2e76-b10c-4182-a146-27e2881af01c@github.com>
 <1ju9rkwe3zB5-d2gv4lGv4ZhwCQSzzGjEbPO9TQpeoU=.0bb33e71-1729-42d8-81b0-bc4a4e7b678a@github.com>
Message-ID: <kVnvT1_351vjXlc5qAqXIr0_8DPa-SaauOe7dcn_feA=.ba42f488-b997-4d2d-8817-ef0c5740a9b8@github.com>

On Fri, 15 Nov 2024 09:20:39 GMT, Ludvig Janiuk <lujaniuk at openjdk.org> wrote:

>> The way this feature selected "the full set of tests" was actually getting "the set of tests to be run". But often, we run some subset through either specifying test files, folders, or groups. Our problemlist files will still have entries for other tests we were not running right now.
>> 
>> As a solution, --verify-exclude needs to compute the maximal set of tests that could be run for all test suites that any of the tests being run are a part of. I've implemented this by creating a dummy testmanager and passing no parameters to it.
>> 
>> Additionally, the regex for problemlist format had a typo.
>
> Ludvig Janiuk has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - verbose output
>  - rm todo

Looks good to me.

-------------

Marked as reviewed by cstein (Committer).

PR Review: https://git.openjdk.org/jtreg/pull/236#pullrequestreview-2448204709

From cstein at openjdk.org  Wed Nov 20 11:10:49 2024
From: cstein at openjdk.org (Christian Stein)
Date: Wed, 20 Nov 2024 11:10:49 GMT
Subject: RFR: 7903872: Option testThreadFactoryPath support relativepath
 [v3]
In-Reply-To: <Riq7Ngt4gIFQvFNF0u_yqwRrKFTANhugxkhbgJoZyzY=.5a5c5857-9f22-4bd9-ad37-dd9af16d9c28@github.com>
References: <N3KRfHTVirffwmi4f2aHN0pLQO4vVbtRAd2ZEKpV7NU=.150de2c8-cde9-4bab-89a8-de2b344d90e6@github.com>
 <Riq7Ngt4gIFQvFNF0u_yqwRrKFTANhugxkhbgJoZyzY=.5a5c5857-9f22-4bd9-ad37-dd9af16d9c28@github.com>
Message-ID: <GCRNPOs1YV3im4woF7mL4cVJhp2qHaLtq4dRFBI2Gaw=.08fd8282-461f-43bd-a55d-319a798e5015@github.com>

On Mon, 21 Oct 2024 16:38:38 GMT, SendaoYan <syan at openjdk.org> wrote:

>> Hi all,
>> Currently, the jtreg option `-testThreadFactoryPath` doesn't support relativepath.
>> I think jtreg can support relativepath for option `-testThreadFactoryPath`, same like to `-jdk` or `-nativepath`.
>> Change has been verified locally, risk is low.
>
> SendaoYan has updated the pull request incrementally with one additional commit since the last revision:
> 
>   add testcase TestThreadFactoryRelativePath.ok

Looks good to me.

-------------

Marked as reviewed by cstein (Committer).

PR Review: https://git.openjdk.org/jtreg/pull/233#pullrequestreview-2448307112

From jpai at openjdk.org  Wed Nov 20 11:39:34 2024
From: jpai at openjdk.org (Jaikiran Pai)
Date: Wed, 20 Nov 2024 11:39:34 GMT
Subject: RFR: 7903872: Option testThreadFactoryPath support relativepath
 [v3]
In-Reply-To: <Riq7Ngt4gIFQvFNF0u_yqwRrKFTANhugxkhbgJoZyzY=.5a5c5857-9f22-4bd9-ad37-dd9af16d9c28@github.com>
References: <N3KRfHTVirffwmi4f2aHN0pLQO4vVbtRAd2ZEKpV7NU=.150de2c8-cde9-4bab-89a8-de2b344d90e6@github.com>
 <Riq7Ngt4gIFQvFNF0u_yqwRrKFTANhugxkhbgJoZyzY=.5a5c5857-9f22-4bd9-ad37-dd9af16d9c28@github.com>
Message-ID: <WcL2YEMDgxz8NrzhzetnLgAdyqg0PrSIIoIYkMGcbHg=.7ec054cd-106f-42db-96e7-0178de6fa1a1@github.com>

On Mon, 21 Oct 2024 16:38:38 GMT, SendaoYan <syan at openjdk.org> wrote:

>> Hi all,
>> Currently, the jtreg option `-testThreadFactoryPath` doesn't support relativepath.
>> I think jtreg can support relativepath for option `-testThreadFactoryPath`, same like to `-jdk` or `-nativepath`.
>> Change has been verified locally, risk is low.
>
> SendaoYan has updated the pull request incrementally with one additional commit since the last revision:
> 
>   add testcase TestThreadFactoryRelativePath.ok

This looks good to me. I haven't had a chance to run the test myself. Please verify that this updated test as well as existing tests continue to pass, before integrating.

-------------

Marked as reviewed by jpai (Reviewer).

PR Review: https://git.openjdk.org/jtreg/pull/233#pullrequestreview-2448370357

From syan at openjdk.org  Wed Nov 20 13:02:36 2024
From: syan at openjdk.org (SendaoYan)
Date: Wed, 20 Nov 2024 13:02:36 GMT
Subject: RFR: 7903872: Option testThreadFactoryPath support relativepath
 [v3]
In-Reply-To: <Riq7Ngt4gIFQvFNF0u_yqwRrKFTANhugxkhbgJoZyzY=.5a5c5857-9f22-4bd9-ad37-dd9af16d9c28@github.com>
References: <N3KRfHTVirffwmi4f2aHN0pLQO4vVbtRAd2ZEKpV7NU=.150de2c8-cde9-4bab-89a8-de2b344d90e6@github.com>
 <Riq7Ngt4gIFQvFNF0u_yqwRrKFTANhugxkhbgJoZyzY=.5a5c5857-9f22-4bd9-ad37-dd9af16d9c28@github.com>
Message-ID: <DhycBjfNU64ZkQTajxbyZTBEgejFVkOdxd6WY22SnYs=.9454b7db-76ee-4f66-9ca3-4a9488099175@github.com>

On Mon, 21 Oct 2024 16:38:38 GMT, SendaoYan <syan at openjdk.org> wrote:

>> Hi all,
>> Currently, the jtreg option `-testThreadFactoryPath` doesn't support relativepath.
>> I think jtreg can support relativepath for option `-testThreadFactoryPath`, same like to `-jdk` or `-nativepath`.
>> Change has been verified locally, risk is low.
>
> SendaoYan has updated the pull request incrementally with one additional commit since the last revision:
> 
>   add testcase TestThreadFactoryRelativePath.ok

Thanks all for the review.

The newly added tests and the existing tests run passed locally.

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/233#issuecomment-2488521420

From duke at openjdk.org  Wed Nov 20 13:02:36 2024
From: duke at openjdk.org (duke)
Date: Wed, 20 Nov 2024 13:02:36 GMT
Subject: RFR: 7903872: Option testThreadFactoryPath support relativepath
 [v3]
In-Reply-To: <Riq7Ngt4gIFQvFNF0u_yqwRrKFTANhugxkhbgJoZyzY=.5a5c5857-9f22-4bd9-ad37-dd9af16d9c28@github.com>
References: <N3KRfHTVirffwmi4f2aHN0pLQO4vVbtRAd2ZEKpV7NU=.150de2c8-cde9-4bab-89a8-de2b344d90e6@github.com>
 <Riq7Ngt4gIFQvFNF0u_yqwRrKFTANhugxkhbgJoZyzY=.5a5c5857-9f22-4bd9-ad37-dd9af16d9c28@github.com>
Message-ID: <fNi75dJjVQ9UycyLfWW6EShUI9hZ_ZLkaAx1Q4inyWY=.81c93b62-03b1-40c7-9891-516a6c9e88fb@github.com>

On Mon, 21 Oct 2024 16:38:38 GMT, SendaoYan <syan at openjdk.org> wrote:

>> Hi all,
>> Currently, the jtreg option `-testThreadFactoryPath` doesn't support relativepath.
>> I think jtreg can support relativepath for option `-testThreadFactoryPath`, same like to `-jdk` or `-nativepath`.
>> Change has been verified locally, risk is low.
>
> SendaoYan has updated the pull request incrementally with one additional commit since the last revision:
> 
>   add testcase TestThreadFactoryRelativePath.ok

@sendaoYan 
Your change (at version 193923a47dd75213774cbc19839da5c8e78bca9a) is now ready to be sponsored by a Committer.

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/233#issuecomment-2488524535

From duke at openjdk.org  Thu Nov 21 19:26:28 2024
From: duke at openjdk.org (andrlos)
Date: Thu, 21 Nov 2024 19:26:28 GMT
Subject: RFR: 7903519 : jtreg/jtharness is missing features for basic crash
 testing
In-Reply-To: <n9SzF-5HeuT0XXmYS8lIxl3he5aZBRlM1Tl5b0fYI-M=.af90f2ed-5784-49bf-9faa-d26dbcfaf747@github.com>
References: <1k_V7Eg7B_QW6-WNSNhWWezgOgfLFLnS5zK85TfTiZ0=.b616aac4-079f-4023-adb6-7e4542c7d856@github.com>
 <uA9y4ZKCN_-Rsri91_WEPdahbaZOPe6W-fn5YzhP7QA=.d1764c3f-eab1-4689-b0e5-22d0ead28a18@github.com>
 <8Vqx8LlIfN9WFhOFOpF7v1dAul4balD5EdGktzip9hA=.f6a673f6-0089-41ec-8241-8cc5f01bcfb5@github.com>
 <n9SzF-5HeuT0XXmYS8lIxl3he5aZBRlM1Tl5b0fYI-M=.af90f2ed-5784-49bf-9faa-d26dbcfaf747@github.com>
Message-ID: <uQgoBEE7yNXQ5jAnDZWM2mn3dGaj9qmdeniktBudKlk=.6df6ee83-59c2-4780-88bf-38bd701e5641@github.com>

On Tue, 12 Nov 2024 09:53:17 GMT, Christian Stein <cstein at openjdk.org> wrote:

>> @sormuras 	feast your eyes on the code below :D
>> 
>> 
>> public class CrashOnlyStatusTransformer implements StatusTransformer {
>>         @Override
>>         public Status transform(Status originalStatus, TestDescription td) {
>>             Status newStatus = originalStatus;
>>             if(originalStatus.getType() == Status.FAILED && ! this.didCrash(td)){
>>                 newStatus = new Status(Status.PASSED, "Just a regular failure.");
>>             }
>>             return newStatus;
>>         }
>> 
>>         private boolean didCrash(TestDescription td){
>>             Pattern pattern = Pattern.compile("^hs_err_pid(\\d+)\.log");
>>             List<String> hs_errs = Arrays.stream(td.getDir().list()).filter(pattern.asPredicate()).collect(Collectors.toList());
>>             return !hs_errs.isEmpty();
>>     }
>> }
>> 
>> 
>> this is an approach that we use for crashtesting with debug jdk builds to separate crashes from regular failures
>
>> [...] this is an approach that we use for crashtesting with debug jdk builds to separate crashes from regular failures
> 
> And sometimes tests do produce `hs_err_pid.log` files and expect/assert them; marking those tests as `PASSED`, right?
> 
> Where (console log, web-view, ...) do you check for such crashes? Manually or with tool/script support?
> 
> Isn't it possible to implement/apply an after-the-fact filter that doesn't rewrite actual run results?

@sormuras 	@jaikiran I would like to also point out, since Junit has been already mentioned here, that JUnit since JUnit4 provides a feature called TestRule where you can implement a custom testRule, that allows the user to modify behavior of any test.. It can be used in a similar scenario, by implementing a TestRule that ignores any regular exceptions, thus only reporting JVM crashes as failures. https://github.com/junit-team/junit4/blob/HEAD/doc/ReleaseNotes4.9.md

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/235#issuecomment-2492076652

From syan at openjdk.org  Fri Nov 22 04:16:29 2024
From: syan at openjdk.org (SendaoYan)
Date: Fri, 22 Nov 2024 04:16:29 GMT
Subject: Integrated: 7903872: Option testThreadFactoryPath support relativepath
In-Reply-To: <N3KRfHTVirffwmi4f2aHN0pLQO4vVbtRAd2ZEKpV7NU=.150de2c8-cde9-4bab-89a8-de2b344d90e6@github.com>
References: <N3KRfHTVirffwmi4f2aHN0pLQO4vVbtRAd2ZEKpV7NU=.150de2c8-cde9-4bab-89a8-de2b344d90e6@github.com>
Message-ID: <_J9n6nYNNwKS835CwSFdEvrDO7VrlR1PkAgi2Ua3UX8=.a21d7969-c3e1-4c7a-b35c-ea44bf9cbf54@github.com>

On Mon, 21 Oct 2024 09:48:08 GMT, SendaoYan <syan at openjdk.org> wrote:

> Hi all,
> Currently, the jtreg option `-testThreadFactoryPath` doesn't support relativepath.
> I think jtreg can support relativepath for option `-testThreadFactoryPath`, same like to `-jdk` or `-nativepath`.
> Change has been verified locally, risk is low.

This pull request has now been integrated.

Changeset: 8e74d559
Author:    SendaoYan <syan at openjdk.org>
Committer: Jaikiran Pai <jpai at openjdk.org>
URL:       https://git.openjdk.org/jtreg/commit/8e74d559375769f9c6e4c82970f7207173c547f8
Stats:     31 lines in 2 files changed: 27 ins; 0 del; 4 mod

7903872: Option testThreadFactoryPath support relativepath

Reviewed-by: cstein, jpai

-------------

PR: https://git.openjdk.org/jtreg/pull/233

From syan at openjdk.org  Fri Nov 22 05:52:27 2024
From: syan at openjdk.org (SendaoYan)
Date: Fri, 22 Nov 2024 05:52:27 GMT
Subject: RFR: 7903872: Option testThreadFactoryPath support relativepath
 [v3]
In-Reply-To: <Riq7Ngt4gIFQvFNF0u_yqwRrKFTANhugxkhbgJoZyzY=.5a5c5857-9f22-4bd9-ad37-dd9af16d9c28@github.com>
References: <N3KRfHTVirffwmi4f2aHN0pLQO4vVbtRAd2ZEKpV7NU=.150de2c8-cde9-4bab-89a8-de2b344d90e6@github.com>
 <Riq7Ngt4gIFQvFNF0u_yqwRrKFTANhugxkhbgJoZyzY=.5a5c5857-9f22-4bd9-ad37-dd9af16d9c28@github.com>
Message-ID: <DZ5Ca47KzkOu1zq09dko07qGPzTHzIpjIu4QP_c0__Q=.cee25518-3eee-4fff-a13e-4229081e4942@github.com>

On Mon, 21 Oct 2024 16:38:38 GMT, SendaoYan <syan at openjdk.org> wrote:

>> Hi all,
>> Currently, the jtreg option `-testThreadFactoryPath` doesn't support relativepath.
>> I think jtreg can support relativepath for option `-testThreadFactoryPath`, same like to `-jdk` or `-nativepath`.
>> Change has been verified locally, risk is low.
>
> SendaoYan has updated the pull request incrementally with one additional commit since the last revision:
> 
>   add testcase TestThreadFactoryRelativePath.ok

Thanks for the sponsor.

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/233#issuecomment-2492923565