RFR (S): 8006413: Add utility classes for writing better multiprocess tests in jtreg

[Joe Darcy]

Dmitry,

On 1/17/2013 9:58 AM, Dmitry Samersoff wrote:
> Kelly,
>
> We have couple of platform depended points that makes shell tests not
> reliable:
>
> 1. Getting PID of right process
> 2. Kill running process
> 3. Remove remaining locks and locked files/directories (e.g. hsperfdata)

There are many more shell-specific problems too in terms of differences 
in what is accepted by the shell we run on different platforms, 
differences in behavior and location of commands, etc.

> etc
>
> Java has all these problems as well, so I can't imagine why Java based
> tests (I speak about multiprocess one only) is more reliable than shell
> ones.

Well, we the JDK group can and has written and shipped cross-platform 
functionality for such tasks as part of the JDK libraries.  We should 
feel free to use that work in JDK testing too.

-Joe

>
>    The simplest way to improve reliability of shell based tests is write
> different scripts for windows and for other platforms.
>    On my opinion most of problem with shell scripts is the result of
> attempt to use unix style programming under windows. (e.g. tlist works
> much better than ps from MKS or Cygwin)
>
> -Dmitry
>
>
> On 2013-01-17 21:14, Kelly O'Hair wrote:
>> Yes the devil is in shell scripts. Yes it's a pain to program client/server tests with pure Java,
>> but with pure Java you can share that logic much safer than shell scripts.
>> It's even more painful to get client/server tests to work reliably in shell scripts, on all platforms.
>>
>> Shell tests have been so problematic for so long that we need to start forcing the issue,
>> 'no more shell tests' should now be the rule.
>> Developers need to figure out how to share Java client/server logic between tests and stop adding
>> more to this shell test nightmare.
>>
>> -kto
>>
>> On Jan 17, 2013, at 9:02 AM, Dmitry Samersoff wrote:
>>
>>> Joe,
>>>
>>> The devil is not in shell scripts.  The devil is in the fact that
>>> harness (jtreg) doesn't have a process management and doesn't support
>>> client server tests.
>>>
>>> Shell initially designed to manage processes but java not, so replacing
>>> shell scripts to java based process builder that exec java from within
>>> java that exec another java doesn't make things better and just leads to
>>> the new layer of errors.
>>>
>>> I guess we should invest some time and efforts to jtreg and finaly being
>>> able to write something like (approximately)
>>>
>>> @run/server/timeout=300 someClass
>>> @run/client someClass
>>>
>>> and then use
>>> jtreg.getServerPid(),
>>> jtreg.shutdownServer()
>>>
>>> etc.
>>>
>>> Then I vote with two hands for getting rid of all shell based tests.
>>>
>>> But today I see no benefits from replacing
>>>
>>> java -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -version |
>>>   grep "warning: Using incremental CMS is deprecated"
>>>
>>> to a 16 lines of Java code (
>>> http://cr.openjdk.java.net/~brutisso/8006398/webrev.00/test/gc/startup-warnings/TestCMSIncrementalMode.java.html
>>> )
>>>
>>>
>>> -Dmitry
>>>
>>>
>>>
>>>
>>> On 2013-01-17 20:34, Joe Darcy wrote:
>>>> On 1/17/2013 7:15 AM, Christian Törnqvist wrote:
>>>>>> What is benefits of this library over usual shell script?
>>>>> It works on all platforms, also the code is a lot cleaner and easier
>>>>> to maintain.
>>>> Shell tests are awful. We should strive to not only stop adding more
>>>> shell tests, I think we should work to convert existing shell tests into
>>>> Java programs.
>>>>
>>>> I've been looking a bit at the set of tests with transient failures are
>>>> shell tests are overrepresented on that list.  Writing cross platform
>>>> shell code is tricky and the shells on some platforms have a history of
>>>> being at least one of slow and unreliable. Additionally, the test
>>>> themselves often have bugs.  For example, Alan Bateman just recently
>>>> pushed a changest to have the tests properly respect environment
>>>> variables passed in by jtreg [1].
>>>>
>>>> -Joe
>>>>
>>>> [1] 8005978: shell tests need to use the $COMPILEJDK for javac, jar and
>>>> other tools
>>>> http://hg.openjdk.java.net/jdk8/tl/jdk/rev/7da291690aa0
>>>>
>>>>> Thanks,
>>>>> Christian
>>>>>
>>>>> -----Original Message-----
>>>>> From: Dmitry Samersoff
>>>>> Sent: den 17 januari 2013 15:59
>>>>> To: Christian Törnqvist
>>>>> Cc: Coleen Phillimore; hotspot-dev at openjdk.java.net
>>>>> Subject: Re: RFR (S): 8006413: Add utility classes for writing better
>>>>> multiprocess tests in jtreg
>>>>>
>>>>> On 2013-01-17 18:10, Christian Törnqvist wrote:
>>>>>> Hi Dmitry,
>>>>>>
>>>>>>> 1. I'm second to Coleen. Is it possible to add more comments
>>>>>>> explaining how to use this tools.
>>>>>> My intention is to write a detailed guide to writing jtreg tests,
>>>>>> including the use of these (and future) utility classes.
>>>>> Good.
>>>>>
>>>>>>> 2. Is it possible to create your own exception rather than use
>>>>>>> Exceptioin class directly.
>>>>>> I've changed it to use RuntimeException based on feedback from other
>>>>>> people
>>>>> Please create your own exception class. I would like to be able to
>>>>> distinguish between exceptions that comes from different parts of tests.
>>>>>
>>>>>>> 3. Who is responsible to kill started process?
>>>>>> This is really a task for the jtreg framework. Honestly it doesn't
>>>>>> really do this very well right now, since the process handling in Java
>>>>>> is not good enough to deal with this. I think this is a something we
>>>>>> should look into addressing in jtreg rather than having the tests
>>>>>> worry about it.
>>>>> What is benefits of this library over usual shell script?
>>>>>
>>>>> -Dmitry
>>>>>
>>>>>
>>>>>> Best regards, Christian
>>>>>>
>>>>>> -----Original Message----- From: Dmitry Samersoff Sent: den 17 januari
>>>>>> 2013 14:14 To: Coleen Phillimore Cc:
>>>>>> hotspot-dev at openjdk.java.net Subject: Re: RFR (S): 8006413: Add
>>>>>> utility classes for writing better multiprocess tests in jtreg
>>>>>>
>>>>>> Christian,
>>>>>>
>>>>>> 1. I'm second to Coleen. Is it possible to add more comments
>>>>>> explaining how to use this tools.
>>>>>>
>>>>>>
>>>>>> 2. Is it possible to create your own exception rather than use
>>>>>> Exceptioin class directly.
>>>>>>
>>>>>>
>>>>>> 3. Who is responsible to kill started process?
>>>>>>
>>>>>> -Dmitry
>>>>>>
>>>>>>
>>>>>> On 2013-01-16 17:53, Coleen Phillimore wrote:
>>>>>>> Christian, Can you write up (somewhere!) what these are for and how
>>>>>>> these are going to be used?  I don't understand just from this
>>>>>>> code.   Does jtreg know enough to build these classfiles?   Do we
>>>>>>> have to write shell scripts to refer to these? Thanks, Coleen
>>>>>>>
>>>>>>> On 1/16/2013 7:34 AM, Christian Törnqvist wrote:
>>>>>>>> Hi everyone,
>>>>>>>>
>>>>>>>>
>>>>>>>> This RFE adds a few utility classes to make it a bit easier to write
>>>>>>>> multi-process tests in jtreg, webrev can be found at
>>>>>>>> http://cr.openjdk.java.net/~brutisso/8006413/webrev.00/
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Christian
>>>>>> -- Dmitry Samersoff Oracle Java development team, Saint Petersburg,
>>>>>> Russia * Give Rabbit time, and he'll always get the answer
>>>>>>
>>>>> -- 
>>>>> Dmitry Samersoff
>>>>> Oracle Java development team, Saint Petersburg, Russia
>>>>> * Give Rabbit time, and he'll always get the answer
>>>
>>> -- 
>>> Dmitry Samersoff
>>> Oracle Java development team, Saint Petersburg, Russia
>>> * Give Rabbit time, and he'll always get the answer
>