RFR (S): 8006413: Add utility classes for writing better multiprocess tests in jtreg

[Dmitry Samersoff]

Jonathan,

Just to clarify, I propose:

1. Add TestTools directory to JTREG workspace.
2. Build TestTools.jar together with jtreg
3. Maintain and distribute TestTools.jar together with
   jtreg under jtreg policies.
4. Add <jtreg distribution>/TestTools.jar to default classpath for
   all tests.
5. Use import com.oracle.jtreg.tools.* when necessary.

-Dmitry



On 2013-01-17 21:23, Jonathan Gibbons wrote:
> I think there's a confusion here between an external dependency on the
> test harness (which has generally been deemed acceptable) and a
> potentially new external dependencies on test libraries in other
> repositories in the forest, which would see less acceptable to me.
> 
> There /are/ times when cut-n-paste is the correct solution. :-)
> 
> -- Jon
> 
> On 01/17/2013 09:17 AM, Kelly O'Hair wrote:
>> The C compiler, bash, libraries, unix utilities, aren't in the repo.
>> We will have and always have
>> external dependencies. I agree we need to limit them, but a testing
>> harness (ideally one) is a very
>> valid external dependency to have.
>>
>> Hotspot needs to get lined up with the standard jdk testing, jtreg is
>> already used in  jdk, JPRT,
>> QE teams, and many developers are very familiar with it.
>>
>> -kto
>>
>> On Jan 17, 2013, at 6:10 AM, Christian Törnqvist wrote:
>>
>>> Well, jtreg is not in the hotspot repo (and shouldn't be) and
>>> suddenly you end up having an external dependency. I'd like to keep
>>> the tests free from dependence on jtreg (along with other external
>>> resources).
>>>
>>> Thanks,
>>> Christian
>>>
>>> -----Original Message-----
>>> From: Dmitry Samersoff
>>> Sent: den 17 januari 2013 15:06
>>> To: Christian Törnqvist
>>> Cc: Stuart Marks; hotspot-dev at openjdk.java.net; Jonathan Gibbons
>>> Subject: Re: RFR (S): 8006413: Add utility classes for writing better
>>> multiprocess tests in jtreg
>>>
>>> Christian,
>>>
>>> On my opinion something like
>>>
>>> import com.oracle.jtreg.tools.*
>>>
>>> is less painful than maintain several independent implementations of
>>> the same ideas.
>>>
>>> -Dmitry
>>>
>>>
>>> On 2013-01-17 18:01, Christian Törnqvist wrote:
>>>>> Is it possible to have a library like this one included into jtreg
>>>>> and available for everybody ?
>>>> The problem with this is that suddenly the tests have a dependency on
>>>> jtreg. One (of many) nice things with the jtreg tests is that they can
>>>> be run as a normal java program without the need to involve jtreg,
>>>> making debugging and troubleshooting them a lot easier.
>>>>
>>>> Best regards, Christian
>>>>
>>>> -----Original Message----- From: Dmitry Samersoff Sent: den 17 januari
>>>> 2013 14:20 To: Stuart Marks Cc: Christian Törnqvist;
>>>> hotspot-dev at openjdk.java.net; Jonathan Gibbons Subject: Re: RFR (S):
>>>> 8006413: Add utility classes for writing better multiprocess tests in
>>>> jtreg
>>>>
>>>> Stuart,
>>>>
>>>>> I work on the JDK core libraries. It turns out that there is a
>>>>> similar library used for testing RMI.
>>>> I guess if we take close look to exising tests we would find couple
>>>> more copies of the same ideas.
>>>>
>>>> Is it possible to have a library like this one included into jtreg and
>>>> available for everybody ?
>>>>
>>>> -Dmitry
>>>>
>>>>
>>>> On 2013-01-17 01:54, Stuart Marks wrote:
>>>>> Hi everybody,
>>>>>
>>>>> I work on the JDK core libraries. It turns out that there is a
>>>>> similar library used for testing RMI. Take a look at
>>>>> jdk/test/java/rmi/testlibrary. It's been around for quite some time
>>>>> already, but I've been doing some maintenance on it recently.
>>>>> (Also, it still needs more work.)
>>>>>
>>>>> Since the RMI library is in the jdk repo and these test utilities are
>>>>> in hotspot, it seems like sharing of code might be difficult.
>>>>> But we might want to compare notes. Here are some observations and
>>>>> comments (though not an actual code review).
>>>>>
>>>>> * The use of a pair of threads (StreamPumper) to collect output from
>>>>> the subprocess is very similar to what RMI does. See the StreamPipe
>>>>> class in RMI's test library, which are used by the JavaVM utility
>>>>> class. One thing I did do recently was to make sure that the
>>>>> subprocess is waited for and the threads are joined. This hadn't been
>>>>> done before, leading to race conditions where some subprocess output
>>>>> was dropped.
>>>>>
>>>>> * Since these threads are specifically joined, I'm not sure of the
>>>>> utility of making them daemon threads.
>>>>>
>>>>> * My style in writing test code and test library code is to be quite
>>>>> verbose about everything and to check every possible error condition.
>>>>> Nothing is worse than seeing a test fail and not having enough
>>>>> information in the log to find out why. Actually, I can think of
>>>>> something worse: having a test pass erroneously because some error
>>>>> condition wasn't checked. (Yes, this has happened.)
>>>>>
>>>>> * Another style point is that I think it's OK for test library code
>>>>> just to throw exceptions out to the caller. My test code usually has
>>>>> 'throws IOException' or even 'throws Exception' on most of the
>>>>> methods, including main, so having test library code throw checked
>>>>> exceptions is reasonable. Jtreg will catch exceptions thrown from
>>>>> main and log them. Usually you want tests to fail quickly at the
>>>>> point of error. So, I concur with Vitaly's concern about ProcessTools
>>>>> handling of IOException and about letting interrupts propagate.
>>>>>
>>>>> * Actually I have a suggestion from Mandy Chung for RMI's StreamPipe
>>>>> threads to catch exceptions and report them back to their "parent"
>>>>> JavaVM. This is probably a rare occurrence but it does ensure that
>>>>> any errors that occur here won't go unnoticed. See
>>>>> JDK-8005827 (
>>>>> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=8005827 )
>>>>>
>>>>> * Regarding jtreg and test library code, it's pretty simple to use.
>>>>> In each test that uses the library code, one adds the @library tag to
>>>>> declare the library. It's probably a good idea to add an @build tag
>>>>> to build each library class that's used in the test. Things usually
>>>>> work in the absence of @build tags, because of implicit compilation,
>>>>> but it's kind of fragile.
>>>>>
>>>>> Anyway, I just wanted to share some notes and ideas.
>>>>>
>>>>> s'marks
>>>>>
>>>>>
>>>>> On 1/16/13 4:34 AM, Christian Törnqvist wrote:
>>>>>> Hi everyone,
>>>>>>
>>>>>> ?
>>>>>>
>>>>>> This RFE adds a few utility classes to make it a bit easier to write
>>>>>> multi-process tests in jtreg, webrev can be found at
>>>>>> http://cr.openjdk.java.net/~brutisso/8006413/webrev.00/
>>>>>>
>>>>>> ?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Christian
>>>>>>
>>>>
>>>> -- Dmitry Samersoff Oracle Java development team, Saint Petersburg,
>>>> Russia * Give Rabbit time, and he'll always get the answer
>>>>
>>>
>>> -- 
>>> Dmitry Samersoff
>>> Oracle Java development team, Saint Petersburg, Russia
>>> * Give Rabbit time, and he'll always get the answer
> 


-- 
Dmitry Samersoff
Oracle Java development team, Saint Petersburg, Russia
* Give Rabbit time, and he'll always get the answer