From jai.forums2013 at gmail.com  Mon Nov  1 03:22:35 2021
From: jai.forums2013 at gmail.com (Jaikiran Pai)
Date: Mon, 1 Nov 2021 08:52:35 +0530
Subject: jtreg adds additional backslashes in Strings that are written to
 System.out/err?
In-Reply-To: <58984f20-bea1-528e-9040-18909605d733@oracle.com>
References: <30a84ffd-7f37-3440-c820-34f2d02cf8f5@gmail.com>
 <0af0cecb-6ef4-b8d5-684b-5d1bb0cec1ad@gmail.com>
 <58984f20-bea1-528e-9040-18909605d733@oracle.com>
Message-ID: <603d7933-3eba-0f58-fb2d-6c2b51903e40@gmail.com>

Hello Jon,

Sorry about the delayed reply. I wanted to give these options a try 
before replying, but never got to it until now.

On 15/09/21 8:00 pm, Jonathan Gibbons wrote:
> Jaikiran,
>
> The encoding is intentional ... and has been there since the beginning 
> of jtreg!
>
> While .jtr files are intended to only contain ASCII characters, they 
> are not intended for direct human consumption. 

Understood. Given that these jtr files get linked and rendered in the 
html output/report, I was under the impression that this is for human 
consumption.

> The stream output from tests is encoded so that it can be subsequently 
> read back in by jtreg and related software.
>
> To view the output of a test without the encoding used in the .jtr 
> file, use one of:
>
> ?* run the test under jtreg with the -va option
> ?* run the test directly, using its main(String...) method ... you can
> ?? get a script to run the test from the "rerun" section in the .jtr file
> ?* use jtreg to report the content of the stream using the
> ?? `-show:<name>` option, such as `jtreg -w:/path/to/work
> ?? -show:System.out /path/to/test

For me, both -va and -show option worked fine. I haven't exactly tested 
this against the backslash issue, but at least I now know what's the 
recommended way to consume this output.

-Jaikiran


From jai.forums2013 at gmail.com  Sat Nov 13 03:39:37 2021
From: jai.forums2013 at gmail.com (Jaikiran Pai)
Date: Sat, 13 Nov 2021 09:09:37 +0530
Subject: jtreg cannot be run against a pre-built/downloaded JDK anymore?
Message-ID: <5405e633-00b4-949f-a982-d1057187d21a@gmail.com>

In order to reproduce one of the issues I have been looking into, I've 
been trying to run a jtreg test case against a Java 17 installation. The 
command I use is:

java -jar jtreg.jar -jdk:<path-to-jdk-17-home> 
test/jdk/java/..../SomeTest.java

This runs into the following exception:

failed to get value for vm.cds.write.archived.java.heap
java.lang.UnsatisfiedLinkError: 'boolean 
jdk.test.whitebox.WhiteBox.canWriteJavaHeapArchive()'
 ?? ?at jdk.test.whitebox.WhiteBox.canWriteJavaHeapArchive(Native Method)
 ?? ?at requires.VMProps.vmCDSCanWriteArchivedJavaHeap(VMProps.java:413)
 ?? ?at requires.VMProps$SafeMap.put(VMProps.java:72)
 ?? ?at requires.VMProps.call(VMProps.java:113)
 ?? ?at requires.VMProps.call(VMProps.java:60)
 ?? ?at 
com.sun.javatest.regtest.agent.GetJDKProperties.run(GetJDKProperties.java:80)
 ?? ?at 
com.sun.javatest.regtest.agent.GetJDKProperties.main(GetJDKProperties.java:54)
Test results: failed: 1

Is this something I am doing wrong or is it some genuine issue? I 
haven't been able to run jtreg against a downloaded/installed JDK for 
many weeks now. Initially I thought I had somehow messed my local jdk 
source repo setup so didn't pay much attention to the failures. But now, 
I'm trying this on a completely different clean setup and that too runs 
into this issue.

Here's the output of jtreg -version:

jtreg 6.1-dev+1
Installed in <some-location>\jtreg\lib\jtreg.jar
Running on platform version 17.0.1 from <some-location>\jdk-17.0.1.
Built with Java(TM) 2 SDK, Version 1.8.0_312-b07 on November 12, 2021.
Copyright (c) 1999, 2021, Oracle and/or its affiliates. All rights reserved.
Use is subject to license terms.
JT Harness, version 6.0 ea b14 (November 12, 2021)
JCov 3.0-2
Java Assembler Tools, version 7.0 ea b09 (November 12, 2021)
TestNG (testng.jar): version 7.3.0
TestNG (jcommander.jar): version unknown
TestNG (guice.jar): version 4.2.3
JUnit (junit.jar): version 4.13.2
JUnit (hamcrest.jar): version 2.2

-Jaikiran




From leonid.mesnik at oracle.com  Sat Nov 13 04:08:34 2021
From: leonid.mesnik at oracle.com (Leonid Mesnik)
Date: Sat, 13 Nov 2021 04:08:34 +0000
Subject: jtreg cannot be run against a pre-built/downloaded JDK anymore?
In-Reply-To: <5405e633-00b4-949f-a982-d1057187d21a@gmail.com>
References: <5405e633-00b4-949f-a982-d1057187d21a@gmail.com>
Message-ID: <BYAPR10MB2471FBE391C0C48115E285839F969@BYAPR10MB2471.namprd10.prod.outlook.com>

Hi

It is a hotpost testing problem rather than a jtreg problem. So I?ve added hotspot-dev at openjdk.java.net<mailto:hotspot-dev at openjdk.java.net> alias.

Seems that problem is that WhiteBox API used in  testing doesn?t correspond to JDK being tested.

This commit changed WhiteBox.canWriteJavaHeapArchive() method
https://github.com/openjdk/jdk/commit/922e86f4ff28c7b17af8e7b5867a40fc76b7fdd7#diff-e75d116b35afd951f114c2b0793b26d0009b441653d6b28d611afcbe0106dfd0

So  you might see this linkage error if tries to test older version of JDK while tests have these changes.

Could you please check that you use exactly the same sources during testing which have been used to build JDK.

Leonid

From: jtreg-use <jtreg-use-retn at openjdk.java.net> on behalf of Jaikiran Pai <jai.forums2013 at gmail.com>
Date: Friday, November 12, 2021 at 8:40 PM
To: jtreg-use at openjdk.java.net <jtreg-use at openjdk.java.net>
Subject: jtreg cannot be run against a pre-built/downloaded JDK anymore?
In order to reproduce one of the issues I have been looking into, I've
been trying to run a jtreg test case against a Java 17 installation. The
command I use is:

java -jar jtreg.jar -jdk:<path-to-jdk-17-home>
test/jdk/java/..../SomeTest.java

This runs into the following exception:

failed to get value for vm.cds.write.archived.java.heap
java.lang.UnsatisfiedLinkError: 'boolean
jdk.test.whitebox.WhiteBox.canWriteJavaHeapArchive()'
     at jdk.test.whitebox.WhiteBox.canWriteJavaHeapArchive(Native Method)
     at requires.VMProps.vmCDSCanWriteArchivedJavaHeap(VMProps.java:413)
     at requires.VMProps$SafeMap.put(VMProps.java:72)
     at requires.VMProps.call(VMProps.java:113)
     at requires.VMProps.call(VMProps.java:60)
     at
com.sun.javatest.regtest.agent.GetJDKProperties.run(GetJDKProperties.java:80)
     at
com.sun.javatest.regtest.agent.GetJDKProperties.main(GetJDKProperties.java:54)
Test results: failed: 1

Is this something I am doing wrong or is it some genuine issue? I
haven't been able to run jtreg against a downloaded/installed JDK for
many weeks now. Initially I thought I had somehow messed my local jdk
source repo setup so didn't pay much attention to the failures. But now,
I'm trying this on a completely different clean setup and that too runs
into this issue.

Here's the output of jtreg -version:

jtreg 6.1-dev+1
Installed in <some-location>\jtreg\lib\jtreg.jar
Running on platform version 17.0.1 from <some-location>\jdk-17.0.1.
Built with Java(TM) 2 SDK, Version 1.8.0_312-b07 on November 12, 2021.
Copyright (c) 1999, 2021, Oracle and/or its affiliates. All rights reserved.
Use is subject to license terms.
JT Harness, version 6.0 ea b14 (November 12, 2021)
JCov 3.0-2
Java Assembler Tools, version 7.0 ea b09 (November 12, 2021)
TestNG (testng.jar): version 7.3.0
TestNG (jcommander.jar): version unknown
TestNG (guice.jar): version 4.2.3
JUnit (junit.jar): version 4.13.2
JUnit (hamcrest.jar): version 2.2

-Jaikiran


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/jtreg-use/attachments/20211113/55e1d410/attachment.htm>

From jai.forums2013 at gmail.com  Sat Nov 13 04:56:28 2021
From: jai.forums2013 at gmail.com (Jaikiran Pai)
Date: Sat, 13 Nov 2021 10:26:28 +0530
Subject: jtreg cannot be run against a pre-built/downloaded JDK anymore?
In-Reply-To: <BYAPR10MB2471FBE391C0C48115E285839F969@BYAPR10MB2471.namprd10.prod.outlook.com>
References: <5405e633-00b4-949f-a982-d1057187d21a@gmail.com>
 <BYAPR10MB2471FBE391C0C48115E285839F969@BYAPR10MB2471.namprd10.prod.outlook.com>
Message-ID: <57f314df-33fd-1af1-8468-19d17b6d69ad@gmail.com>

Hello Leonid,

On 13/11/21 9:38 am, Leonid Mesnik wrote:
> Hi
>
> It is a hotpost testing problem rather than a jtreg problem. So I?ve added hotspot-dev at openjdk.java.net<mailto:hotspot-dev at openjdk.java.net> alias.
Thank you for adding the right list.
> ...
> Could you please check that you use exactly the same sources during testing which have been used to build JDK.

Do you mean the sources of the JDK against which the test is being run? 
I don't have those sources since this test runs against a pre-built 
binary downloaded from https://jdk.java.net/17/

-Jaikiran


From jai.forums2013 at gmail.com  Sat Nov 13 05:37:46 2021
From: jai.forums2013 at gmail.com (Jaikiran Pai)
Date: Sat, 13 Nov 2021 11:07:46 +0530
Subject: jtreg cannot be run against a pre-built/downloaded JDK anymore?
In-Reply-To: <57f314df-33fd-1af1-8468-19d17b6d69ad@gmail.com>
References: <5405e633-00b4-949f-a982-d1057187d21a@gmail.com>
 <BYAPR10MB2471FBE391C0C48115E285839F969@BYAPR10MB2471.namprd10.prod.outlook.com>
 <57f314df-33fd-1af1-8468-19d17b6d69ad@gmail.com>
Message-ID: <680a2669-d5f8-a596-e3ad-071b9fb66233@gmail.com>

I got past this with an extensive workaround for now. I moved/copied 
that test case java file outside of the JDK source tree, then created a 
new/custom TEST.ROOT which is very minimal and has no reference to 
whitebox for bootlibs, then made sure the jtwork directory is also 
outside of the JDK source tree (so that the test is compiled afresh) and 
then ran that test. That helped, but it's only for this test since its 
requirements in the test are very minimal. I don't see a way to get past 
this if I have to run the wider range of jtreg tests that reside in the 
JDK source tree against a pre-built/downloaded Java 17 or any previous 
versions.

-Jaikiran

On 13/11/21 10:26 am, Jaikiran Pai wrote:
> Hello Leonid,
>
> On 13/11/21 9:38 am, Leonid Mesnik wrote:
>> Hi
>>
>> It is a hotpost testing problem rather than a jtreg problem. So I?ve 
>> added 
>> hotspot-dev at openjdk.java.net<mailto:hotspot-dev at openjdk.java.net> alias.
> Thank you for adding the right list.
>> ...
>> Could you please check that you use exactly the same sources during 
>> testing which have been used to build JDK.
>
> Do you mean the sources of the JDK against which the test is being 
> run? I don't have those sources since this test runs against a 
> pre-built binary downloaded from https://jdk.java.net/17/
>
> -Jaikiran
>

From david.holmes at oracle.com  Sat Nov 13 06:37:53 2021
From: david.holmes at oracle.com (David Holmes)
Date: Sat, 13 Nov 2021 16:37:53 +1000
Subject: jtreg cannot be run against a pre-built/downloaded JDK anymore?
In-Reply-To: <680a2669-d5f8-a596-e3ad-071b9fb66233@gmail.com>
References: <5405e633-00b4-949f-a982-d1057187d21a@gmail.com>
 <BYAPR10MB2471FBE391C0C48115E285839F969@BYAPR10MB2471.namprd10.prod.outlook.com>
 <57f314df-33fd-1af1-8468-19d17b6d69ad@gmail.com>
 <680a2669-d5f8-a596-e3ad-071b9fb66233@gmail.com>
Message-ID: <1102c99b-e4e7-5862-fd81-cb39af4c3a81@oracle.com>

On 13/11/2021 3:37 pm, Jaikiran Pai wrote:
> I got past this with an extensive workaround for now. I moved/copied 
> that test case java file outside of the JDK source tree, then created a 
> new/custom TEST.ROOT which is very minimal and has no reference to 
> whitebox for bootlibs, then made sure the jtwork directory is also 
> outside of the JDK source tree (so that the test is compiled afresh) and 
> then ran that test. That helped, but it's only for this test since its 
> requirements in the test are very minimal. I don't see a way to get past 
> this if I have to run the wider range of jtreg tests that reside in the 
> JDK source tree against a pre-built/downloaded Java 17 or any previous 
> versions.

Basically you're not supposed to do that. You have to test a given 
binary with the tests that existed when that binary was built. Many 
things in the tests can change that will fail to run with an older JDK.

In theory you can use the build number of the binary JDK to checkout the 
tests corresponding to that build using the appropriate build tag.

Cheers,
David

> -Jaikiran
> 
> On 13/11/21 10:26 am, Jaikiran Pai wrote:
>> Hello Leonid,
>>
>> On 13/11/21 9:38 am, Leonid Mesnik wrote:
>>> Hi
>>>
>>> It is a hotpost testing problem rather than a jtreg problem. So I?ve 
>>> added 
>>> hotspot-dev at openjdk.java.net<mailto:hotspot-dev at openjdk.java.net> alias.
>> Thank you for adding the right list.
>>> ...
>>> Could you please check that you use exactly the same sources during 
>>> testing which have been used to build JDK.
>>
>> Do you mean the sources of the JDK against which the test is being 
>> run? I don't have those sources since this test runs against a 
>> pre-built binary downloaded from https://jdk.java.net/17/
>>
>> -Jaikiran
>>

From thomas.stuefe at gmail.com  Sat Nov 13 07:56:40 2021
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Sat, 13 Nov 2021 08:56:40 +0100
Subject: jtreg cannot be run against a pre-built/downloaded JDK anymore?
In-Reply-To: <1102c99b-e4e7-5862-fd81-cb39af4c3a81@oracle.com>
References: <5405e633-00b4-949f-a982-d1057187d21a@gmail.com>
 <BYAPR10MB2471FBE391C0C48115E285839F969@BYAPR10MB2471.namprd10.prod.outlook.com>
 <57f314df-33fd-1af1-8468-19d17b6d69ad@gmail.com>
 <680a2669-d5f8-a596-e3ad-071b9fb66233@gmail.com>
 <1102c99b-e4e7-5862-fd81-cb39af4c3a81@oracle.com>
Message-ID: <CAA-vtUzt=efM-HRZP3cM1tBFbGWjZj-XGjtCZa6dC3kS5NZ7yQ@mail.gmail.com>

Maybe the easiest way for you would be to get the source drop matching the
binary JDK from the vendor of your JDK. Since you may also have
vendor-specific changes (albeit rare, its possible).

Cheers, Thomas


On Sat, Nov 13, 2021 at 7:38 AM David Holmes <david.holmes at oracle.com>
wrote:

> On 13/11/2021 3:37 pm, Jaikiran Pai wrote:
> > I got past this with an extensive workaround for now. I moved/copied
> > that test case java file outside of the JDK source tree, then created a
> > new/custom TEST.ROOT which is very minimal and has no reference to
> > whitebox for bootlibs, then made sure the jtwork directory is also
> > outside of the JDK source tree (so that the test is compiled afresh) and
> > then ran that test. That helped, but it's only for this test since its
> > requirements in the test are very minimal. I don't see a way to get past
> > this if I have to run the wider range of jtreg tests that reside in the
> > JDK source tree against a pre-built/downloaded Java 17 or any previous
> > versions.
>
> Basically you're not supposed to do that. You have to test a given
> binary with the tests that existed when that binary was built. Many
> things in the tests can change that will fail to run with an older JDK.
>
> In theory you can use the build number of the binary JDK to checkout the
> tests corresponding to that build using the appropriate build tag.
>
> Cheers,
> David
>
> > -Jaikiran
> >
> > On 13/11/21 10:26 am, Jaikiran Pai wrote:
> >> Hello Leonid,
> >>
> >> On 13/11/21 9:38 am, Leonid Mesnik wrote:
> >>> Hi
> >>>
> >>> It is a hotpost testing problem rather than a jtreg problem. So I?ve
> >>> added
> >>> hotspot-dev at openjdk.java.net<mailto:hotspot-dev at openjdk.java.net>
> alias.
> >> Thank you for adding the right list.
> >>> ...
> >>> Could you please check that you use exactly the same sources during
> >>> testing which have been used to build JDK.
> >>
> >> Do you mean the sources of the JDK against which the test is being
> >> run? I don't have those sources since this test runs against a
> >> pre-built binary downloaded from https://jdk.java.net/17/
> >>
> >> -Jaikiran
> >>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/jtreg-use/attachments/20211113/c468efb6/attachment-0001.htm>

From jai.forums2013 at gmail.com  Sat Nov 13 08:28:14 2021
From: jai.forums2013 at gmail.com (Jaikiran Pai)
Date: Sat, 13 Nov 2021 13:58:14 +0530
Subject: jtreg cannot be run against a pre-built/downloaded JDK anymore?
In-Reply-To: <1102c99b-e4e7-5862-fd81-cb39af4c3a81@oracle.com>
References: <5405e633-00b4-949f-a982-d1057187d21a@gmail.com>
 <BYAPR10MB2471FBE391C0C48115E285839F969@BYAPR10MB2471.namprd10.prod.outlook.com>
 <57f314df-33fd-1af1-8468-19d17b6d69ad@gmail.com>
 <680a2669-d5f8-a596-e3ad-071b9fb66233@gmail.com>
 <1102c99b-e4e7-5862-fd81-cb39af4c3a81@oracle.com>
Message-ID: <1fb7457b-d933-836a-5f97-f078a320ab72@gmail.com>

Hello David,

On 13/11/21 12:07 pm, David Holmes wrote:
> On 13/11/2021 3:37 pm, Jaikiran Pai wrote:
>> I got past this with an extensive workaround for now. I moved/copied 
>> that test case java file outside of the JDK source tree, then created 
>> a new/custom TEST.ROOT which is very minimal and has no reference to 
>> whitebox for bootlibs, then made sure the jtwork directory is also 
>> outside of the JDK source tree (so that the test is compiled afresh) 
>> and then ran that test. That helped, but it's only for this test 
>> since its requirements in the test are very minimal. I don't see a 
>> way to get past this if I have to run the wider range of jtreg tests 
>> that reside in the JDK source tree against a pre-built/downloaded 
>> Java 17 or any previous versions.
>
> Basically you're not supposed to do that. 

I wasn't aware of that. I used to use this method to selectively run 
newly added jtreg tests against different downloaded versions of JDK and 
assumed it was a supported usecase.

Thanks everyone for the inputs.

-Jaikiran


From jai.forums2013 at gmail.com  Mon Nov 15 13:45:13 2021
From: jai.forums2013 at gmail.com (Jaikiran Pai)
Date: Mon, 15 Nov 2021 19:15:13 +0530
Subject: jtreg cannot be run against a pre-built/downloaded JDK anymore?
In-Reply-To: <CAA-vtUzt=efM-HRZP3cM1tBFbGWjZj-XGjtCZa6dC3kS5NZ7yQ@mail.gmail.com>
References: <5405e633-00b4-949f-a982-d1057187d21a@gmail.com>
 <BYAPR10MB2471FBE391C0C48115E285839F969@BYAPR10MB2471.namprd10.prod.outlook.com>
 <57f314df-33fd-1af1-8468-19d17b6d69ad@gmail.com>
 <680a2669-d5f8-a596-e3ad-071b9fb66233@gmail.com>
 <1102c99b-e4e7-5862-fd81-cb39af4c3a81@oracle.com>
 <CAA-vtUzt=efM-HRZP3cM1tBFbGWjZj-XGjtCZa6dC3kS5NZ7yQ@mail.gmail.com>
Message-ID: <468cdc88-86de-8fde-a1b7-44837b9453a5@gmail.com>

The way I used to use this previously was more for convenience than 
anything more. Very specifically, I used to do something like this:

- Work on some bug fix with latest JDK master source repo.

- Add a jtreg test to verify the fix

- Send out a PR and wait for reviews

- On some occasions, the review suggestions include relatively big 
changes to the jtreg test case. In such cases, I used to do those 
changes in the test, verify that the test still continues to pass. 
However, I would even want to make sure the test still reproduces the 
original issue. So instead of git reverting only the source code 
changes, building the current JDK again and then running the updated 
test, I would just point the jtreg run to a differently older version of 
a JDK (which wouldn't have the fix) by using the 
-jdk:<path-to-downloaded-jdk>. I would then expect the test to fail with 
the expected issue.

It was just a convenience than anything more.

-Jaikiran

On 13/11/21 1:26 pm, Thomas St?fe wrote:
> Maybe the easiest way for you would be to get the source drop matching the
> binary JDK from the vendor of your JDK. Since you may also have
> vendor-specific changes (albeit rare, its possible).
>
> Cheers, Thomas
>
>
> On Sat, Nov 13, 2021 at 7:38 AM David Holmes <david.holmes at oracle.com>
> wrote:
>
>> On 13/11/2021 3:37 pm, Jaikiran Pai wrote:
>>> I got past this with an extensive workaround for now. I moved/copied
>>> that test case java file outside of the JDK source tree, then created a
>>> new/custom TEST.ROOT which is very minimal and has no reference to
>>> whitebox for bootlibs, then made sure the jtwork directory is also
>>> outside of the JDK source tree (so that the test is compiled afresh) and
>>> then ran that test. That helped, but it's only for this test since its
>>> requirements in the test are very minimal. I don't see a way to get past
>>> this if I have to run the wider range of jtreg tests that reside in the
>>> JDK source tree against a pre-built/downloaded Java 17 or any previous
>>> versions.
>> Basically you're not supposed to do that. You have to test a given
>> binary with the tests that existed when that binary was built. Many
>> things in the tests can change that will fail to run with an older JDK.
>>
>> In theory you can use the build number of the binary JDK to checkout the
>> tests corresponding to that build using the appropriate build tag.
>>
>> Cheers,
>> David
>>
>>> -Jaikiran
>>>
>>> On 13/11/21 10:26 am, Jaikiran Pai wrote:
>>>> Hello Leonid,
>>>>
>>>> On 13/11/21 9:38 am, Leonid Mesnik wrote:
>>>>> Hi
>>>>>
>>>>> It is a hotpost testing problem rather than a jtreg problem. So I?ve
>>>>> added
>>>>> hotspot-dev at openjdk.java.net<mailto:hotspot-dev at openjdk.java.net>
>> alias.
>>>> Thank you for adding the right list.
>>>>> ...
>>>>> Could you please check that you use exactly the same sources during
>>>>> testing which have been used to build JDK.
>>>> Do you mean the sources of the JDK against which the test is being
>>>> run? I don't have those sources since this test runs against a
>>>> pre-built binary downloaded from https://jdk.java.net/17/
>>>>
>>>> -Jaikiran
>>>>

From david.holmes at oracle.com  Mon Nov 15 21:59:28 2021
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 16 Nov 2021 07:59:28 +1000
Subject: jtreg cannot be run against a pre-built/downloaded JDK anymore?
In-Reply-To: <468cdc88-86de-8fde-a1b7-44837b9453a5@gmail.com>
References: <5405e633-00b4-949f-a982-d1057187d21a@gmail.com>
 <BYAPR10MB2471FBE391C0C48115E285839F969@BYAPR10MB2471.namprd10.prod.outlook.com>
 <57f314df-33fd-1af1-8468-19d17b6d69ad@gmail.com>
 <680a2669-d5f8-a596-e3ad-071b9fb66233@gmail.com>
 <1102c99b-e4e7-5862-fd81-cb39af4c3a81@oracle.com>
 <CAA-vtUzt=efM-HRZP3cM1tBFbGWjZj-XGjtCZa6dC3kS5NZ7yQ@mail.gmail.com>
 <468cdc88-86de-8fde-a1b7-44837b9453a5@gmail.com>
Message-ID: <98cb4719-387d-fe60-e368-e9805917dba6@oracle.com>

On 15/11/2021 11:45 pm, Jaikiran Pai wrote:
> The way I used to use this previously was more for convenience than 
> anything more. Very specifically, I used to do something like this:
> 
> - Work on some bug fix with latest JDK master source repo.
> 
> - Add a jtreg test to verify the fix
> 
> - Send out a PR and wait for reviews
> 
> - On some occasions, the review suggestions include relatively big 
> changes to the jtreg test case. In such cases, I used to do those 
> changes in the test, verify that the test still continues to pass. 
> However, I would even want to make sure the test still reproduces the 
> original issue. So instead of git reverting only the source code 
> changes, building the current JDK again and then running the updated 
> test, I would just point the jtreg run to a differently older version of 
> a JDK (which wouldn't have the fix) by using the 
> -jdk:<path-to-downloaded-jdk>. I would then expect the test to fail with 
> the expected issue.
> 
> It was just a convenience than anything more.

Sure and most of the time that will work. But if the test relies on 
something that is only present in the later JDK binary then obviously 
the test will fail.

Cheers,
David

> -Jaikiran
> 
> On 13/11/21 1:26 pm, Thomas St?fe wrote:
>> Maybe the easiest way for you would be to get the source drop matching 
>> the
>> binary JDK from the vendor of your JDK. Since you may also have
>> vendor-specific changes (albeit rare, its possible).
>>
>> Cheers, Thomas
>>
>>
>> On Sat, Nov 13, 2021 at 7:38 AM David Holmes <david.holmes at oracle.com>
>> wrote:
>>
>>> On 13/11/2021 3:37 pm, Jaikiran Pai wrote:
>>>> I got past this with an extensive workaround for now. I moved/copied
>>>> that test case java file outside of the JDK source tree, then created a
>>>> new/custom TEST.ROOT which is very minimal and has no reference to
>>>> whitebox for bootlibs, then made sure the jtwork directory is also
>>>> outside of the JDK source tree (so that the test is compiled afresh) 
>>>> and
>>>> then ran that test. That helped, but it's only for this test since its
>>>> requirements in the test are very minimal. I don't see a way to get 
>>>> past
>>>> this if I have to run the wider range of jtreg tests that reside in the
>>>> JDK source tree against a pre-built/downloaded Java 17 or any previous
>>>> versions.
>>> Basically you're not supposed to do that. You have to test a given
>>> binary with the tests that existed when that binary was built. Many
>>> things in the tests can change that will fail to run with an older JDK.
>>>
>>> In theory you can use the build number of the binary JDK to checkout the
>>> tests corresponding to that build using the appropriate build tag.
>>>
>>> Cheers,
>>> David
>>>
>>>> -Jaikiran
>>>>
>>>> On 13/11/21 10:26 am, Jaikiran Pai wrote:
>>>>> Hello Leonid,
>>>>>
>>>>> On 13/11/21 9:38 am, Leonid Mesnik wrote:
>>>>>> Hi
>>>>>>
>>>>>> It is a hotpost testing problem rather than a jtreg problem. So I?ve
>>>>>> added
>>>>>> hotspot-dev at openjdk.java.net<mailto:hotspot-dev at openjdk.java.net>
>>> alias.
>>>>> Thank you for adding the right list.
>>>>>> ...
>>>>>> Could you please check that you use exactly the same sources during
>>>>>> testing which have been used to build JDK.
>>>>> Do you mean the sources of the JDK against which the test is being
>>>>> run? I don't have those sources since this test runs against a
>>>>> pre-built binary downloaded from https://jdk.java.net/17/
>>>>>
>>>>> -Jaikiran
>>>>>

From jai.forums2013 at gmail.com  Sun Nov 21 10:00:05 2021
From: jai.forums2013 at gmail.com (Jaikiran Pai)
Date: Sun, 21 Nov 2021 15:30:05 +0530
Subject: Centrally set a system property that gets passed to all tests in a
 sub-directory?
Message-ID: <5af89c48-d098-745f-f90f-f434bd8dc508@gmail.com>

Is there a way in jtreg where a particular Java system property can be 
passed to all tests that run in othervm mode and belong to a particular 
sub-directory in the test root?

Something like "othervm.test.java.opts"? What I'm trying to solve is 
instead of editing each of the 10+ test classes that have a @run 
testng/othervm or @run main/othervm to include -Dfoo=bar in the test 
definition, I would like to have a TEST.properties for that 
sub-directory which would contain this system property to be set while 
launching those tests? I have been looking the documentation 
https://openjdk.java.net/jtreg/tag-spec.html#config but couldn't find 
anything that would help in this case.


-Jaikiran


From jonathan.gibbons at oracle.com  Sun Nov 21 15:38:46 2021
From: jonathan.gibbons at oracle.com (Jonathan Gibbons)
Date: Sun, 21 Nov 2021 07:38:46 -0800
Subject: Centrally set a system property that gets passed to all tests in
 a sub-directory?
In-Reply-To: <5af89c48-d098-745f-f90f-f434bd8dc508@gmail.com>
References: <5af89c48-d098-745f-f90f-f434bd8dc508@gmail.com>
Message-ID: <cdbf6f19-e3ce-002e-7eef-548e60dbe051@oracle.com>

Sorry, there is no such feature.

-- Jon

On 11/21/21 2:00 AM, Jaikiran Pai wrote:
> Is there a way in jtreg where a particular Java system property can be 
> passed to all tests that run in othervm mode and belong to a 
> particular sub-directory in the test root?
>
> Something like "othervm.test.java.opts"? What I'm trying to solve is 
> instead of editing each of the 10+ test classes that have a @run 
> testng/othervm or @run main/othervm to include -Dfoo=bar in the test 
> definition, I would like to have a TEST.properties for that 
> sub-directory which would contain this system property to be set while 
> launching those tests? I have been looking the documentation 
> https://openjdk.java.net/jtreg/tag-spec.html#config but couldn't find 
> anything that would help in this case.
>
>
> -Jaikiran
>

From jai.forums2013 at gmail.com  Mon Nov 22 03:22:43 2021
From: jai.forums2013 at gmail.com (Jaikiran Pai)
Date: Mon, 22 Nov 2021 08:52:43 +0530
Subject: Centrally set a system property that gets passed to all tests in
 a sub-directory?
In-Reply-To: <cdbf6f19-e3ce-002e-7eef-548e60dbe051@oracle.com>
References: <5af89c48-d098-745f-f90f-f434bd8dc508@gmail.com>
 <cdbf6f19-e3ce-002e-7eef-548e60dbe051@oracle.com>
Message-ID: <daf4bebc-2716-b67a-a037-9aa9b703fe6c@gmail.com>

Thank you Jon.

The original use case where I felt the need for this was in context of 
https://bugs.openjdk.java.net/browse/JDK-8277507 where I wanted to 
introduce this diagnostic system property for launched test cases under 
a specific test directory. Given that the issue affected all the tests 
in that directory (there were around 20 tests spread across 2 
sub-directories), I didn't want to change each one of them separately.

I ended up using a different way to achieve this, but that's mainly 
because I realized later that these tests were launching a separate 
process (using ProcessBuilder) after they were launched by jtreg. So 
this ability to add this property centrally from a TEST.properties 
wouldn't have helped anyway. But I think this might be a useful 
enhancement in general. Do you think this is worth filing a JBS enhancement?

-Jaikiran

On 21/11/21 9:08 pm, Jonathan Gibbons wrote:
> Sorry, there is no such feature.
>
> -- Jon
>
> On 11/21/21 2:00 AM, Jaikiran Pai wrote:
>> Is there a way in jtreg where a particular Java system property can 
>> be passed to all tests that run in othervm mode and belong to a 
>> particular sub-directory in the test root?
>>
>> Something like "othervm.test.java.opts"? What I'm trying to solve is 
>> instead of editing each of the 10+ test classes that have a @run 
>> testng/othervm or @run main/othervm to include -Dfoo=bar in the test 
>> definition, I would like to have a TEST.properties for that 
>> sub-directory which would contain this system property to be set 
>> while launching those tests? I have been looking the documentation 
>> https://openjdk.java.net/jtreg/tag-spec.html#config but couldn't find 
>> anything that would help in this case.
>>
>>
>> -Jaikiran
>>

From jonathan.gibbons at oracle.com  Mon Nov 22 16:52:00 2021
From: jonathan.gibbons at oracle.com (Jonathan Gibbons)
Date: Mon, 22 Nov 2021 08:52:00 -0800
Subject: [External] : Re: Centrally set a system property that gets passed
 to all tests in a sub-directory?
In-Reply-To: <daf4bebc-2716-b67a-a037-9aa9b703fe6c@gmail.com>
References: <5af89c48-d098-745f-f90f-f434bd8dc508@gmail.com>
 <cdbf6f19-e3ce-002e-7eef-548e60dbe051@oracle.com>
 <daf4bebc-2716-b67a-a037-9aa9b703fe6c@gmail.com>
Message-ID: <75de5178-060b-a08e-d742-f762100d5797@oracle.com>

Your solution (about using ProcessBuilder) indirectly suggests that the 
ability to centrally set a system property does not really need to be in 
jtreg.

Instead of using `@run main/othervm ...` to run a test, and having jtreg 
set system properties for you, you can use `@run driver` which is 
similar to `@run main` except that it does not start an extra JVM. This 
allows a test to run any custom code and start the JVM itself with all 
the necessary desired characteristics.? Obviously, this can be done with 
a test library, if that is convenient, and that library could set up 
system properties and environment variables and any shared command-line 
options.

-- Jon


On 11/21/21 7:22 PM, Jaikiran Pai wrote:
> Thank you Jon.
>
> The original use case where I felt the need for this was in context of 
> https://bugs.openjdk.java.net/browse/JDK-8277507 where I wanted to 
> introduce this diagnostic system property for launched test cases 
> under a specific test directory. Given that the issue affected all the 
> tests in that directory (there were around 20 tests spread across 2 
> sub-directories), I didn't want to change each one of them separately.
>
> I ended up using a different way to achieve this, but that's mainly 
> because I realized later that these tests were launching a separate 
> process (using ProcessBuilder) after they were launched by jtreg. So 
> this ability to add this property centrally from a TEST.properties 
> wouldn't have helped anyway. But I think this might be a useful 
> enhancement in general. Do you think this is worth filing a JBS 
> enhancement?
>
> -Jaikiran
>
> On 21/11/21 9:08 pm, Jonathan Gibbons wrote:
>> Sorry, there is no such feature.
>>
>> -- Jon
>>
>> On 11/21/21 2:00 AM, Jaikiran Pai wrote:
>>> Is there a way in jtreg where a particular Java system property can 
>>> be passed to all tests that run in othervm mode and belong to a 
>>> particular sub-directory in the test root?
>>>
>>> Something like "othervm.test.java.opts"? What I'm trying to solve is 
>>> instead of editing each of the 10+ test classes that have a @run 
>>> testng/othervm or @run main/othervm to include -Dfoo=bar in the test 
>>> definition, I would like to have a TEST.properties for that 
>>> sub-directory which would contain this system property to be set 
>>> while launching those tests? I have been looking the documentation 
>>> https://openjdk.java.net/jtreg/tag-spec.html#config but couldn't 
>>> find anything that would help in this case.
>>>
>>>
>>> -Jaikiran
>>>