jtreg shell tests

[Jonathan Gibbons]

I don't think we should force the elimination of shell tests, but we 
should definitely encourage it. (i.e. Option 1.)

The underlying theme in options 1-4 is to minimize the effort required 
to accommodate the changes needed to support the use of WSL to run 
tests; not to impose effort. Carrots, not sticks.

Of course, Sergey, if you'd like to volunteer to convert all the client 
tests, that would be your prerogative. Just yesterday, I had a chat with 
Phil, giving anecdotes about how we converted almost all of the 
langtools shell tests, by writing a library that provided methods based 
on the shell commands that we saw in our shell scripts: cp, mv, rm, 
diff, grep, etc. Others have done the same for some of the core-libs 
tests. We can start a separate thread if you'd like to discuss such 
techniques.

-- Jon


On 1/25/19 5:16 PM, Sergey Bylokhov wrote:
> No my point was radical drop of all such tests.
>
> On 25/01/2019 17:05, Andrew Luo wrote:
>> Isn't that option 1?
>>
>> Thanks,
>>
>> -Andrew
>>
>> -----Original Message-----
>> From: quality-discuss <quality-discuss-bounces at openjdk.java.net> On 
>> Behalf Of Sergey Bylokhov
>> Sent: Friday, January 25, 2019 5:00 PM
>> To: Jonathan Gibbons <jonathan.gibbons at oracle.com>; 
>> quality-discuss at openjdk.java.net
>> Subject: Re: jtreg shell tests
>>
>> There is one more Option 5.
>>
>> Drop shell tests from the workspace and provide some examples on how 
>> to write such logic using java api.
>>
>> On 25/01/2019 16:43, Jonathan Gibbons wrote:
>>> With all the recent discussion regarding how to support the use of
>>> Windows Subsystem for Linux (WSL) as an alternate to Cygwin, it 
>>> seems worth writing up some recommendations on writing jtreg shell 
>>> tests.
>>> The intent of these notes is that they will evolve into a page in 
>>> the jtreg section on the OpenJDK website.
>>>
>>> The focus is specifically about different approaches to providing the
>>> ability to run a shell test on all supported platforms, by means of
>>> abstracting the significant differences into a series of environment 
>>> variables that are set according to the environment in which the 
>>> test is running.
>>>
>>> Option 1.
>>>
>>> Convert the test to Java. In general, this continues to be the 
>>> recommended alternative.
>>>
>>>
>>> Option 2.
>>>
>>> Use a shell `case` statement, like the following, or a variant thereof:
>>>
>>>      OS=`uname -s`;
>>>      case "$OS" in
>>>           Windows* | CYGWIN* )
>>>               FS="\\"
>>>               PS=";"
>>>               NULL=NUL
>>>               ;;
>>>
>>>           Linux )
>>>               if [ -r $TESTJAVA/bin/java.exe ]; then
>>>                   FS="\\"
>>>                   PS=";"
>>>                   EXE_SUFFIX=".exe"
>>>               else
>>>                   FS="/"
>>>                   PS=":"
>>>               fi
>>>               NULL=/dev/null
>>>               ;;
>>>
>>>           * )
>>>               FS="/"
>>>               PS=":"
>>>               NULL=/dev/null
>>>      esac
>>>
>>> Option 3.
>>>
>>> Use a shared library script to embody the behavior in the previous 
>>> example.  jtreg now provides a new `TESTROOT` environment variable, 
>>> which makes it easy to reference a shared script in a constant 
>>> manner from any shell test, wherever the test is within the test 
>>> suite. Since the library script is used to set environment variables 
>>> like `FS`, `PS`, and `NULL`, it should be executed with `source` and 
>>> not `bash` or `sh`.
>>>
>>>
>>> Option 4.
>>>
>>> jtreg now sets the following environment variables when running a 
>>> shell script: `FS`, `PS`, `NULL` and `EXE_SUFFIX`. This may be 
>>> enough to completely avoid the need for a `case` statement in each 
>>> shell script or the use of a shared library script to set these 
>>> variables.
>>>
>>>
>>> Running scripts standalone.
>>>
>>> One concern when working with shell tests has been the ability to 
>>> run the test "stand-alone", without the use of jtreg. In the past, 
>>> this was seen as justification for the explicit use of the `case` 
>>> statement in each shell test. However, the need to run shell tests 
>>> standalone no longer seems to be a significant concern. For those 
>>> that do want to run shell tests by themselves, it is worth noting 
>>> that once a test has been run by jtreg, the ".jtr" file contains 
>>> "rerun" sections with details on how to run each action of the test. 
>>> You can either copy/paste/edit from the ".jtr" file directly, or use 
>>> the jtreg `-show:rerun` option to output the information to the 
>>> standard output stream.
>>>
>>>
>>>
>>
>>
>> -- 
>> Best regards, Sergey.
>>
>
>