RFR (S): 8006413: Add utility classes for writing better multiprocess tests in jtreg

[Kelly O'Hair]

Yes the devil is in shell scripts. Yes it's a pain to program client/server tests with pure Java,
but with pure Java you can share that logic much safer than shell scripts.
It's even more painful to get client/server tests to work reliably in shell scripts, on all platforms.

Shell tests have been so problematic for so long that we need to start forcing the issue,
'no more shell tests' should now be the rule.
Developers need to figure out how to share Java client/server logic between tests and stop adding
more to this shell test nightmare.

-kto

On Jan 17, 2013, at 9:02 AM, Dmitry Samersoff wrote:

> Joe,
> 
> The devil is not in shell scripts.  The devil is in the fact that
> harness (jtreg) doesn't have a process management and doesn't support
> client server tests.
> 
> Shell initially designed to manage processes but java not, so replacing
> shell scripts to java based process builder that exec java from within
> java that exec another java doesn't make things better and just leads to
> the new layer of errors.
> 
> I guess we should invest some time and efforts to jtreg and finaly being
> able to write something like (approximately)
> 
> @run/server/timeout=300 someClass
> @run/client someClass
> 
> and then use
> jtreg.getServerPid(),
> jtreg.shutdownServer()
> 
> etc.
> 
> Then I vote with two hands for getting rid of all shell based tests.
> 
> But today I see no benefits from replacing
> 
> java -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -version |
>  grep "warning: Using incremental CMS is deprecated"
> 
> to a 16 lines of Java code (
> http://cr.openjdk.java.net/~brutisso/8006398/webrev.00/test/gc/startup-warnings/TestCMSIncrementalMode.java.html
> )
> 
> 
> -Dmitry
> 
> 
> 
> 
> On 2013-01-17 20:34, Joe Darcy wrote:
>> On 1/17/2013 7:15 AM, Christian Törnqvist wrote:
>>>> What is benefits of this library over usual shell script?
>>> It works on all platforms, also the code is a lot cleaner and easier
>>> to maintain.
>> 
>> Shell tests are awful. We should strive to not only stop adding more
>> shell tests, I think we should work to convert existing shell tests into
>> Java programs.
>> 
>> I've been looking a bit at the set of tests with transient failures are
>> shell tests are overrepresented on that list.  Writing cross platform
>> shell code is tricky and the shells on some platforms have a history of
>> being at least one of slow and unreliable. Additionally, the test
>> themselves often have bugs.  For example, Alan Bateman just recently
>> pushed a changest to have the tests properly respect environment
>> variables passed in by jtreg [1].
>> 
>> -Joe
>> 
>> [1] 8005978: shell tests need to use the $COMPILEJDK for javac, jar and
>> other tools
>> http://hg.openjdk.java.net/jdk8/tl/jdk/rev/7da291690aa0
>> 
>>> 
>>> Thanks,
>>> Christian
>>> 
>>> -----Original Message-----
>>> From: Dmitry Samersoff
>>> Sent: den 17 januari 2013 15:59
>>> To: Christian Törnqvist
>>> Cc: Coleen Phillimore; hotspot-dev at openjdk.java.net
>>> Subject: Re: RFR (S): 8006413: Add utility classes for writing better
>>> multiprocess tests in jtreg
>>> 
>>> On 2013-01-17 18:10, Christian Törnqvist wrote:
>>>> Hi Dmitry,
>>>> 
>>>>> 1. I'm second to Coleen. Is it possible to add more comments
>>>>> explaining how to use this tools.
>>>> My intention is to write a detailed guide to writing jtreg tests,
>>>> including the use of these (and future) utility classes.
>>> Good.
>>> 
>>>>> 2. Is it possible to create your own exception rather than use
>>>>> Exceptioin class directly.
>>>> I've changed it to use RuntimeException based on feedback from other
>>>> people
>>> Please create your own exception class. I would like to be able to
>>> distinguish between exceptions that comes from different parts of tests.
>>> 
>>>>> 3. Who is responsible to kill started process?
>>>> This is really a task for the jtreg framework. Honestly it doesn't
>>>> really do this very well right now, since the process handling in Java
>>>> is not good enough to deal with this. I think this is a something we
>>>> should look into addressing in jtreg rather than having the tests
>>>> worry about it.
>>> What is benefits of this library over usual shell script?
>>> 
>>> -Dmitry
>>> 
>>> 
>>>> Best regards, Christian
>>>> 
>>>> -----Original Message----- From: Dmitry Samersoff Sent: den 17 januari
>>>> 2013 14:14 To: Coleen Phillimore Cc:
>>>> hotspot-dev at openjdk.java.net Subject: Re: RFR (S): 8006413: Add
>>>> utility classes for writing better multiprocess tests in jtreg
>>>> 
>>>> Christian,
>>>> 
>>>> 1. I'm second to Coleen. Is it possible to add more comments
>>>> explaining how to use this tools.
>>>> 
>>>> 
>>>> 2. Is it possible to create your own exception rather than use
>>>> Exceptioin class directly.
>>>> 
>>>> 
>>>> 3. Who is responsible to kill started process?
>>>> 
>>>> -Dmitry
>>>> 
>>>> 
>>>> On 2013-01-16 17:53, Coleen Phillimore wrote:
>>>>> Christian, Can you write up (somewhere!) what these are for and how
>>>>> these are going to be used?  I don't understand just from this
>>>>> code.   Does jtreg know enough to build these classfiles?   Do we
>>>>> have to write shell scripts to refer to these? Thanks, Coleen
>>>>> 
>>>>> On 1/16/2013 7:34 AM, Christian Törnqvist wrote:
>>>>>> Hi everyone,
>>>>>> 
>>>>>> 
>>>>>> This RFE adds a few utility classes to make it a bit easier to write
>>>>>> multi-process tests in jtreg, webrev can be found at
>>>>>> http://cr.openjdk.java.net/~brutisso/8006413/webrev.00/
>>>>>> 
>>>>>> 
>>>>>> Thanks,
>>>>>> 
>>>>>> Christian
>>>> 
>>>> -- Dmitry Samersoff Oracle Java development team, Saint Petersburg,
>>>> Russia * Give Rabbit time, and he'll always get the answer
>>>> 
>>> 
>>> -- 
>>> Dmitry Samersoff
>>> Oracle Java development team, Saint Petersburg, Russia
>>> * Give Rabbit time, and he'll always get the answer
>> 
> 
> 
> -- 
> Dmitry Samersoff
> Oracle Java development team, Saint Petersburg, Russia
> * Give Rabbit time, and he'll always get the answer