From duke at openjdk.org  Thu Dec  5 00:52:51 2024
From: duke at openjdk.org (andrlos)
Date: Thu, 5 Dec 2024 00:52:51 GMT
Subject: RFR: 7903519 : jtreg/jtharness is missing features for basic crash
 testing
In-Reply-To: <1k_V7Eg7B_QW6-WNSNhWWezgOgfLFLnS5zK85TfTiZ0=.b616aac4-079f-4023-adb6-7e4542c7d856@github.com>
References: <1k_V7Eg7B_QW6-WNSNhWWezgOgfLFLnS5zK85TfTiZ0=.b616aac4-079f-4023-adb6-7e4542c7d856@github.com>
Message-ID: <JP3RUp9y8_KL0tVQbPoI9KwKDxKDAlUSvD-cCBJNkWY=.9347d312-f1f1-41e2-8e23-2bfdaa42e814@github.com>

On Wed, 6 Nov 2024 01:45:22 GMT, andrlos <duke at openjdk.org> wrote:

> provides SPI for enabling external status transformations of failed tests
> 
> this is a continuation of efforts after https://github.com/openjdk/jtharness/pull/59 
> 
> Requires newest jtharness build (not even tagged yet) that includes above mentioned change to be compiled succesfully
> 
> The main idea is to provide a unified StatusTransformer interface, that can be externally implemented by users and added to a classpath in a separate jar to allow modifications of test execution status based on some elementary analysis. This can be easily used for crashtesting (filtering out only tests with jvm crashes).

I believe that this approach is pretty minimalistic when all the capabilities for the user are taken into account and presents almost no maintenance overhead. Plus the user of the jtreg project is someone who should know what he is doing and giving him tools to peek inside and modify the test evaluation behavior is nothing bad IMO. Default behavior of jtreg release is also not affected by this change.

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/235#issuecomment-2518853975

From harald+jdklists at volse.no  Fri Dec 13 09:29:24 2024
From: harald+jdklists at volse.no (Harald Eilertsen)
Date: Fri, 13 Dec 2024 10:29:24 +0100
Subject: Support for building jtreg on FreeBSD
Message-ID: <xgngd6ndqqsjurfvn7bryrdolb2by2pqt4tbk5dgabw5g44hwl@d5k2k7re5rjj>

Hi,

I have been contracted by The FreeBSD Foundation to improve Java on
FreeBSD. As part of that I've made some patches to make jtreg build
successfully on that platform, and also pass the self tests.

I'd like to check if there's any chance for getting these changes
included upstream. That would be a great benefit to the further Java on
FreeBSD work.

My patches are here, if anybody would be so kind to have a look:
https://github.com/openjdk/jtreg/compare/master...snake66:jtreg:freebsd-port

The one thing I can think of that may be controversial is that I've made
the makefiles pick up the zip and unzip binaries from $PATH. If you want
to keep the hardcoded absolute paths instead, I'll fix that.

Thanks in advance,

Harald Eilertsen
--
https://kodeknekkeriet.net

From jai.forums2013 at gmail.com  Fri Dec 13 13:37:43 2024
From: jai.forums2013 at gmail.com (Jaikiran Pai)
Date: Fri, 13 Dec 2024 19:07:43 +0530
Subject: Support for building jtreg on FreeBSD
In-Reply-To: <xgngd6ndqqsjurfvn7bryrdolb2by2pqt4tbk5dgabw5g44hwl@d5k2k7re5rjj>
References: <xgngd6ndqqsjurfvn7bryrdolb2by2pqt4tbk5dgabw5g44hwl@d5k2k7re5rjj>
Message-ID: <0bd11aa5-a84b-4b56-9691-6b1d66d96d0b@gmail.com>

Hello Harald,

I haven't checked the patches you listed but contributions to the 
project are welcome. This page 
https://openjdk.org/projects/code-tools/jtreg/ might help you get 
started and this one for OpenJDK contributions in general 
https://openjdk.org/guide/#contributing-to-an-openjdk-project.

I would recommend that you raise individual fixes/changes as separate 
PRs. That way it will be easier to review. Once you initiate the first 
PR, the skara bot will guide you through the OCA process 
(https://openjdk.org/guide/#i-have-a-patch-what-do-i-do) and once the 
OCA is signed we should be able to review those changes.

-Jaikiran

On 13/12/24 2:59 pm, Harald Eilertsen wrote:
> Hi,
>
> I have been contracted by The FreeBSD Foundation to improve Java on
> FreeBSD. As part of that I've made some patches to make jtreg build
> successfully on that platform, and also pass the self tests.
>
> I'd like to check if there's any chance for getting these changes
> included upstream. That would be a great benefit to the further Java on
> FreeBSD work.
>
> My patches are here, if anybody would be so kind to have a look:
> https://github.com/openjdk/jtreg/compare/master...snake66:jtreg:freebsd-port
>
> The one thing I can think of that may be controversial is that I've made
> the makefiles pick up the zip and unzip binaries from $PATH. If you want
> to keep the hardcoded absolute paths instead, I'll fix that.
>
> Thanks in advance,
>
> Harald Eilertsen
> --
> https://kodeknekkeriet.net

From harald+jdklists at volse.no  Sat Dec 14 09:29:03 2024
From: harald+jdklists at volse.no (Harald Eilertsen)
Date: Sat, 14 Dec 2024 10:29:03 +0100
Subject: Support for building jtreg on FreeBSD
In-Reply-To: <0bd11aa5-a84b-4b56-9691-6b1d66d96d0b@gmail.com>
References: <xgngd6ndqqsjurfvn7bryrdolb2by2pqt4tbk5dgabw5g44hwl@d5k2k7re5rjj>
 <0bd11aa5-a84b-4b56-9691-6b1d66d96d0b@gmail.com>
Message-ID: <vnxqdysl6mdt2uw2cnzi6bmrpdlofl4qrpq2ahjcv4bxc23rbe@pjhf2nimcr4j>

Hi Jaikiran,

Thanks for the quick and helpful answer.

On Fri, Dec 13, 2024 at 07:07:43PM +0530, Jaikiran Pai wrote:
> I would recommend that you raise individual fixes/changes as separate PRs.
> That way it will be easier to review.

Thanks, I'll do that.

Take care!
Harald

From jpai at openjdk.org  Mon Dec 16 13:44:56 2024
From: jpai at openjdk.org (Jaikiran Pai)
Date: Mon, 16 Dec 2024 13:44:56 GMT
Subject: RFR: 7903883: --verify-exclude test existence and format checks
 are broken [v2]
In-Reply-To: <1ju9rkwe3zB5-d2gv4lGv4ZhwCQSzzGjEbPO9TQpeoU=.0bb33e71-1729-42d8-81b0-bc4a4e7b678a@github.com>
References: <GTvesx96d3nXWjxlzTX9PxV2UdfLyRgOFNmgJNRlaj8=.59aa2e76-b10c-4182-a146-27e2881af01c@github.com>
 <1ju9rkwe3zB5-d2gv4lGv4ZhwCQSzzGjEbPO9TQpeoU=.0bb33e71-1729-42d8-81b0-bc4a4e7b678a@github.com>
Message-ID: <DX65LkUGMiogEGCV7eH49LOgH7Qib5iXUQZ8EOerebw=.58ff3ea7-e42b-4834-867b-4b98298505df@github.com>

On Fri, 15 Nov 2024 09:20:39 GMT, Ludvig Janiuk <lujaniuk at openjdk.org> wrote:

>> The way this feature selected "the full set of tests" was actually getting "the set of tests to be run". But often, we run some subset through either specifying test files, folders, or groups. Our problemlist files will still have entries for other tests we were not running right now.
>> 
>> As a solution, --verify-exclude needs to compute the maximal set of tests that could be run for all test suites that any of the tests being run are a part of. I've implemented this by creating a dummy testmanager and passing no parameters to it.
>> 
>> Additionally, the regex for problemlist format had a typo.
>
> Ludvig Janiuk has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - verbose output
>  - rm todo

I don't have prior knowledge of this code, but reading through what's stated in the linked JBS issue and a brief look at this code, this change appears reasonable to me.

Please generate a jtreg build with this change and use it against the JDK mainline repo once, to make sure this works as expected and doesn't cause any unforeseen failures.

-------------

Marked as reviewed by jpai (Reviewer).

PR Review: https://git.openjdk.org/jtreg/pull/236#pullrequestreview-2506224638

From jpai at openjdk.org  Mon Dec 16 13:47:52 2024
From: jpai at openjdk.org (Jaikiran Pai)
Date: Mon, 16 Dec 2024 13:47:52 GMT
Subject: RFR: 7903883: --verify-exclude test existence and format checks
 are broken [v2]
In-Reply-To: <1ju9rkwe3zB5-d2gv4lGv4ZhwCQSzzGjEbPO9TQpeoU=.0bb33e71-1729-42d8-81b0-bc4a4e7b678a@github.com>
References: <GTvesx96d3nXWjxlzTX9PxV2UdfLyRgOFNmgJNRlaj8=.59aa2e76-b10c-4182-a146-27e2881af01c@github.com>
 <1ju9rkwe3zB5-d2gv4lGv4ZhwCQSzzGjEbPO9TQpeoU=.0bb33e71-1729-42d8-81b0-bc4a4e7b678a@github.com>
Message-ID: <OpsUa_AVxNjhIhPYzqNddgPQeqXesxegYdE12jk3eWs=.cd0081b9-43d7-4826-aaf2-f637ca579e6a@github.com>

On Fri, 15 Nov 2024 09:20:39 GMT, Ludvig Janiuk <lujaniuk at openjdk.org> wrote:

>> The way this feature selected "the full set of tests" was actually getting "the set of tests to be run". But often, we run some subset through either specifying test files, folders, or groups. Our problemlist files will still have entries for other tests we were not running right now.
>> 
>> As a solution, --verify-exclude needs to compute the maximal set of tests that could be run for all test suites that any of the tests being run are a part of. I've implemented this by creating a dummy testmanager and passing no parameters to it.
>> 
>> Additionally, the regex for problemlist format had a typo.
>
> Ludvig Janiuk has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - verbose output
>  - rm todo

While at it, I noticed in the linked JBS issue, that the current existing error message says:

> The fully qualified test must exists.

The "exists" should have been "exist". Could you update that error message (and related self reg tests) to fix that typo? If you want to do it in a separate PR, that's fine too.

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/236#issuecomment-2545672134

From lujaniuk at openjdk.org  Mon Dec 16 14:25:09 2024
From: lujaniuk at openjdk.org (Ludvig Janiuk)
Date: Mon, 16 Dec 2024 14:25:09 GMT
Subject: RFR: 7903883: --verify-exclude test existence and format checks
 are broken [v3]
In-Reply-To: <GTvesx96d3nXWjxlzTX9PxV2UdfLyRgOFNmgJNRlaj8=.59aa2e76-b10c-4182-a146-27e2881af01c@github.com>
References: <GTvesx96d3nXWjxlzTX9PxV2UdfLyRgOFNmgJNRlaj8=.59aa2e76-b10c-4182-a146-27e2881af01c@github.com>
Message-ID: <gzosJIsWPKviqoZqg1PlBuFS7VvZqbyXX_95zcrXQk8=.8ca45aae-8f24-4aa3-82f6-af62b05ee7d6@github.com>

> The way this feature selected "the full set of tests" was actually getting "the set of tests to be run". But often, we run some subset through either specifying test files, folders, or groups. Our problemlist files will still have entries for other tests we were not running right now.
> 
> As a solution, --verify-exclude needs to compute the maximal set of tests that could be run for all test suites that any of the tests being run are a part of. I've implemented this by creating a dummy testmanager and passing no parameters to it.
> 
> Additionally, the regex for problemlist format had a typo.

Ludvig Janiuk has updated the pull request incrementally with one additional commit since the last revision:

  Exists typo

-------------

Changes:
  - all: https://git.openjdk.org/jtreg/pull/236/files
  - new: https://git.openjdk.org/jtreg/pull/236/files/fd5e203f..b6c5d20d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jtreg&pr=236&range=02
 - incr: https://webrevs.openjdk.org/?repo=jtreg&pr=236&range=01-02

  Stats: 6 lines in 2 files changed: 0 ins; 0 del; 6 mod
  Patch: https://git.openjdk.org/jtreg/pull/236.diff
  Fetch: git fetch https://git.openjdk.org/jtreg.git pull/236/head:pull/236

PR: https://git.openjdk.org/jtreg/pull/236

From jpai at openjdk.org  Mon Dec 16 14:25:09 2024
From: jpai at openjdk.org (Jaikiran Pai)
Date: Mon, 16 Dec 2024 14:25:09 GMT
Subject: RFR: 7903883: --verify-exclude test existence and format checks
 are broken [v3]
In-Reply-To: <gzosJIsWPKviqoZqg1PlBuFS7VvZqbyXX_95zcrXQk8=.8ca45aae-8f24-4aa3-82f6-af62b05ee7d6@github.com>
References: <GTvesx96d3nXWjxlzTX9PxV2UdfLyRgOFNmgJNRlaj8=.59aa2e76-b10c-4182-a146-27e2881af01c@github.com>
 <gzosJIsWPKviqoZqg1PlBuFS7VvZqbyXX_95zcrXQk8=.8ca45aae-8f24-4aa3-82f6-af62b05ee7d6@github.com>
Message-ID: <Ia1iD7dg7yi0fHCQUM02bX7q-OYhV6Gj0pJFwZspMyM=.5e5e585a-bd2e-4808-8e1c-bb076582602b@github.com>

On Mon, 16 Dec 2024 14:21:53 GMT, Ludvig Janiuk <lujaniuk at openjdk.org> wrote:

>> The way this feature selected "the full set of tests" was actually getting "the set of tests to be run". But often, we run some subset through either specifying test files, folders, or groups. Our problemlist files will still have entries for other tests we were not running right now.
>> 
>> As a solution, --verify-exclude needs to compute the maximal set of tests that could be run for all test suites that any of the tests being run are a part of. I've implemented this by creating a dummy testmanager and passing no parameters to it.
>> 
>> Additionally, the regex for problemlist format had a typo.
>
> Ludvig Janiuk has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Exists typo

Marked as reviewed by jpai (Reviewer).

-------------

PR Review: https://git.openjdk.org/jtreg/pull/236#pullrequestreview-2506330049

From lujaniuk at openjdk.org  Mon Dec 16 15:18:52 2024
From: lujaniuk at openjdk.org (Ludvig Janiuk)
Date: Mon, 16 Dec 2024 15:18:52 GMT
Subject: RFR: 7903883: --verify-exclude test existence and format checks
 are broken [v3]
In-Reply-To: <gzosJIsWPKviqoZqg1PlBuFS7VvZqbyXX_95zcrXQk8=.8ca45aae-8f24-4aa3-82f6-af62b05ee7d6@github.com>
References: <GTvesx96d3nXWjxlzTX9PxV2UdfLyRgOFNmgJNRlaj8=.59aa2e76-b10c-4182-a146-27e2881af01c@github.com>
 <gzosJIsWPKviqoZqg1PlBuFS7VvZqbyXX_95zcrXQk8=.8ca45aae-8f24-4aa3-82f6-af62b05ee7d6@github.com>
Message-ID: <29DJsm9sIb7BAsDWCA_Krpmi0hIOZXU-EmB58VAVb_g=.1c4b48fa-16eb-4037-8923-16540a442ea5@github.com>

On Mon, 16 Dec 2024 14:25:09 GMT, Ludvig Janiuk <lujaniuk at openjdk.org> wrote:

>> The way this feature selected "the full set of tests" was actually getting "the set of tests to be run". But often, we run some subset through either specifying test files, folders, or groups. Our problemlist files will still have entries for other tests we were not running right now.
>> 
>> As a solution, --verify-exclude needs to compute the maximal set of tests that could be run for all test suites that any of the tests being run are a part of. I've implemented this by creating a dummy testmanager and passing no parameters to it.
>> 
>> Additionally, the regex for problemlist format had a typo.
>
> Ludvig Janiuk has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Exists typo

I've fixed the exists typo and also ran mainline `configure --with-jtreg=my/build; make test TEST=tier1` with no issues. Therefore integrating.

-------------

PR Comment: https://git.openjdk.org/jtreg/pull/236#issuecomment-2545897583

From lujaniuk at openjdk.org  Mon Dec 16 15:18:52 2024
From: lujaniuk at openjdk.org (Ludvig Janiuk)
Date: Mon, 16 Dec 2024 15:18:52 GMT
Subject: Integrated: 7903883: --verify-exclude test existence and format checks
 are broken
In-Reply-To: <GTvesx96d3nXWjxlzTX9PxV2UdfLyRgOFNmgJNRlaj8=.59aa2e76-b10c-4182-a146-27e2881af01c@github.com>
References: <GTvesx96d3nXWjxlzTX9PxV2UdfLyRgOFNmgJNRlaj8=.59aa2e76-b10c-4182-a146-27e2881af01c@github.com>
Message-ID: <P8Qli38JiuKenFlGue7vDXlVEA2bq7ph2HNGIBBMaoc=.7281fd25-5667-47a7-aa80-7ce81149cbdf@github.com>

On Wed, 13 Nov 2024 09:11:06 GMT, Ludvig Janiuk <lujaniuk at openjdk.org> wrote:

> The way this feature selected "the full set of tests" was actually getting "the set of tests to be run". But often, we run some subset through either specifying test files, folders, or groups. Our problemlist files will still have entries for other tests we were not running right now.
> 
> As a solution, --verify-exclude needs to compute the maximal set of tests that could be run for all test suites that any of the tests being run are a part of. I've implemented this by creating a dummy testmanager and passing no parameters to it.
> 
> Additionally, the regex for problemlist format had a typo.

This pull request has now been integrated.

Changeset: 9bc320a4
Author:    Ludvig Janiuk <lujaniuk at openjdk.org>
URL:       https://git.openjdk.org/jtreg/commit/9bc320a436236d140d13ac25a60e48b3821f9af1
Stats:     137 lines in 7 files changed: 119 ins; 1 del; 17 mod

7903883: --verify-exclude test existence and format checks are broken

Reviewed-by: cstein, jpai

-------------

PR: https://git.openjdk.org/jtreg/pull/236