<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Sergey's suggestion to force the elimination of shell tests may
not be far off the mark. :-)<br>
</p>
<p>It is getting insanely complicated to support maintain these
shell tests. Not impossible, but very, very fragile.</p>
<p>That being said, I have a small demo test suite that mimics the
options 1-4 that I enumerated before, and I have an updated
version of jtreg to run these tests.</p>
<p>This version of jtreg supports the following:</p>
<ul>
<li>bin/jtreg has been updated to work correctly on WSL<br>
<br>
</li>
<li>new option -cygwin, as well as recent new -wsl option, to
force the use of Cygwin, but neither option is necessary in
normal use: jtreg will autodetect whether to use Cygwin or WSL
(or MKS, untested)<br>
<br>
</li>
<li>The following combinations are supported on Windows:</li>
</ul>
<blockquote>
<ul>
<li>Cygwin, jtreg running on a Windows JDK, test JDK is a
Windows JDK.</li>
<li>WSL, jtreg running on a Windows JDK, test JDK is a Windows
JDK.
</li>
<li>WSL, jtreg running on a Linux JDK, test JDK is a Linux JDK.
<br>
</li>
</ul>
</blockquote>
<ul>
<li>jtreg will /not/ support mixed JDKs ... running on one kind of
JDK, and the test JDK is the other kind.<br>
</li>
</ul>
<p><br>
</p>
<p>Part of the reason this is absurdly fragile is because of the
need to use "wslpath" to convert command line args, which (I'm
presuming) we don't want to put in our shell scripts. WSLEnv only
does some of the work; it only converts environment variables; it
does nothing to help fix up command lines. Therefore the only way
to get shell scripts to work is for jtreg to "guess" whether an
environment variable is going to be used by WSL (e.g. a path to
invoke the test JDK) or whether the environment variable is going
to be used by a Windows binary invoked from WSL (e.g. a value for
a classpath option.) <br>
</p>
<p>It is worth noting that we don't have this problem on Cygwin,
because Cygwin will tolerate the use of Windows-style paths, and
so consistent-looking environment variables can be used within the
script and passed to programs. That's not true in the WSL world
at this point.</p>
<p>I need to do more testing on the new version of jtreg; I'll push
changes next week.<br>
</p>
<p>-- Jon<br>
</p>
<br>
<div class="moz-cite-prefix">On 01/26/2019 08:40 AM, Jonathan
Gibbons wrote:<br>
</div>
<blockquote type="cite"
cite="mid:5982df28-04ab-7ea5-aea2-04b9e6c79b29@oracle.com">I don't
think we should force the elimination of shell tests, but we
should definitely encourage it. (i.e. Option 1.)
<br>
<br>
The underlying theme in options 1-4 is to minimize the effort
required to accommodate the changes needed to support the use of
WSL to run tests; not to impose effort. Carrots, not sticks.
<br>
<br>
Of course, Sergey, if you'd like to volunteer to convert all the
client tests, that would be your prerogative. Just yesterday, I
had a chat with Phil, giving anecdotes about how we converted
almost all of the langtools shell tests, by writing a library that
provided methods based on the shell commands that we saw in our
shell scripts: cp, mv, rm, diff, grep, etc. Others have done the
same for some of the core-libs tests. We can start a separate
thread if you'd like to discuss such techniques.
<br>
<br>
-- Jon
<br>
<br>
<br>
On 1/25/19 5:16 PM, Sergey Bylokhov wrote:
<br>
<blockquote type="cite">No my point was radical drop of all such
tests.
<br>
<br>
On 25/01/2019 17:05, Andrew Luo wrote:
<br>
<blockquote type="cite">Isn't that option 1?
<br>
<br>
Thanks,
<br>
<br>
-Andrew
<br>
<br>
-----Original Message-----
<br>
From: quality-discuss
<a class="moz-txt-link-rfc2396E" href="mailto:quality-discuss-bounces@openjdk.java.net"><quality-discuss-bounces@openjdk.java.net></a> On Behalf Of
Sergey Bylokhov
<br>
Sent: Friday, January 25, 2019 5:00 PM
<br>
To: Jonathan Gibbons <a class="moz-txt-link-rfc2396E" href="mailto:jonathan.gibbons@oracle.com"><jonathan.gibbons@oracle.com></a>;
<a class="moz-txt-link-abbreviated" href="mailto:quality-discuss@openjdk.java.net">quality-discuss@openjdk.java.net</a>
<br>
Subject: Re: jtreg shell tests
<br>
<br>
There is one more Option 5.
<br>
<br>
Drop shell tests from the workspace and provide some examples
on how to write such logic using java api.
<br>
<br>
On 25/01/2019 16:43, Jonathan Gibbons wrote:
<br>
<blockquote type="cite">With all the recent discussion
regarding how to support the use of
<br>
Windows Subsystem for Linux (WSL) as an alternate to Cygwin,
it seems worth writing up some recommendations on writing
jtreg shell tests.
<br>
The intent of these notes is that they will evolve into a
page in the jtreg section on the OpenJDK website.
<br>
<br>
The focus is specifically about different approaches to
providing the
<br>
ability to run a shell test on all supported platforms, by
means of
<br>
abstracting the significant differences into a series of
environment variables that are set according to the
environment in which the test is running.
<br>
<br>
Option 1.
<br>
<br>
Convert the test to Java. In general, this continues to be
the recommended alternative.
<br>
<br>
<br>
Option 2.
<br>
<br>
Use a shell `case` statement, like the following, or a
variant thereof:
<br>
<br>
OS=`uname -s`;
<br>
case "$OS" in
<br>
Windows* | CYGWIN* )
<br>
FS="\\"
<br>
PS=";"
<br>
NULL=NUL
<br>
;;
<br>
<br>
Linux )
<br>
if [ -r $TESTJAVA/bin/java.exe ]; then
<br>
FS="\\"
<br>
PS=";"
<br>
EXE_SUFFIX=".exe"
<br>
else
<br>
FS="/"
<br>
PS=":"
<br>
fi
<br>
NULL=/dev/null
<br>
;;
<br>
<br>
* )
<br>
FS="/"
<br>
PS=":"
<br>
NULL=/dev/null
<br>
esac
<br>
<br>
Option 3.
<br>
<br>
Use a shared library script to embody the behavior in the
previous example. jtreg now provides a new `TESTROOT`
environment variable, which makes it easy to reference a
shared script in a constant manner from any shell test,
wherever the test is within the test suite. Since the
library script is used to set environment variables like
`FS`, `PS`, and `NULL`, it should be executed with `source`
and not `bash` or `sh`.
<br>
<br>
<br>
Option 4.
<br>
<br>
jtreg now sets the following environment variables when
running a shell script: `FS`, `PS`, `NULL` and `EXE_SUFFIX`.
This may be enough to completely avoid the need for a `case`
statement in each shell script or the use of a shared
library script to set these variables.
<br>
<br>
<br>
Running scripts standalone.
<br>
<br>
One concern when working with shell tests has been the
ability to run the test "stand-alone", without the use of
jtreg. In the past, this was seen as justification for the
explicit use of the `case` statement in each shell test.
However, the need to run shell tests standalone no longer
seems to be a significant concern. For those that do want to
run shell tests by themselves, it is worth noting that once
a test has been run by jtreg, the ".jtr" file contains
"rerun" sections with details on how to run each action of
the test. You can either copy/paste/edit from the ".jtr"
file directly, or use the jtreg `-show:rerun` option to
output the information to the standard output stream.
<br>
<br>
<br>
<br>
</blockquote>
<br>
<br>
-- <br>
Best regards, Sergey.
<br>
<br>
</blockquote>
<br>
<br>
</blockquote>
</blockquote>
<br>
</body>
</html>