RFR (S): 8006413: Add utility classes for writing better multiprocess tests in jtreg

[Ioi Lam]

How do you ensure that the version of the library included in jtreg is 
in fact consistent with the version of HotSpot/JDK that you are trying 
to build/test?

I have seen that the UTE harness installed at different locations of 
/java have different versions and are configured differently. And that's 
just for 'running' the tests. If we have to include such a dependency 
for 'building' the tests that would be even worse ...

- Ioi

On 01/17/2013 09:17 AM, Kelly O'Hair wrote:
> The C compiler, bash, libraries, unix utilities, aren't in the repo. We will have and always have
> external dependencies. I agree we need to limit them, but a testing harness (ideally one) is a very
> valid external dependency to have.
>
> Hotspot needs to get lined up with the standard jdk testing, jtreg is already used in  jdk, JPRT,
> QE teams, and many developers are very familiar with it.
>
> -kto
>
> On Jan 17, 2013, at 6:10 AM, Christian Törnqvist wrote:
>
>> Well, jtreg is not in the hotspot repo (and shouldn't be) and suddenly you end up having an external dependency. I'd like to keep the tests free from dependence on jtreg (along with other external resources).
>>
>> Thanks,
>> Christian
>>
>> -----Original Message-----
>> From: Dmitry Samersoff
>> Sent: den 17 januari 2013 15:06
>> To: Christian Törnqvist
>> Cc: Stuart Marks; hotspot-dev at openjdk.java.net; Jonathan Gibbons
>> Subject: Re: RFR (S): 8006413: Add utility classes for writing better multiprocess tests in jtreg
>>
>> Christian,
>>
>> On my opinion something like
>>
>> import com.oracle.jtreg.tools.*
>>
>> is less painful than maintain several independent implementations of the same ideas.
>>
>> -Dmitry
>>
>>
>> On 2013-01-17 18:01, Christian Törnqvist wrote:
>>>> Is it possible to have a library like this one included into jtreg
>>>> and available for everybody ?
>>> The problem with this is that suddenly the tests have a dependency on
>>> jtreg. One (of many) nice things with the jtreg tests is that they can
>>> be run as a normal java program without the need to involve jtreg,
>>> making debugging and troubleshooting them a lot easier.
>>>
>>> Best regards, Christian
>>>
>>> -----Original Message----- From: Dmitry Samersoff Sent: den 17 januari
>>> 2013 14:20 To: Stuart Marks Cc: Christian Törnqvist;
>>> hotspot-dev at openjdk.java.net; Jonathan Gibbons Subject: Re: RFR (S):
>>> 8006413: Add utility classes for writing better multiprocess tests in
>>> jtreg
>>>
>>> Stuart,
>>>
>>>> I work on the JDK core libraries. It turns out that there is a
>>>> similar library used for testing RMI.
>>> I guess if we take close look to exising tests we would find couple
>>> more copies of the same ideas.
>>>
>>> Is it possible to have a library like this one included into jtreg and
>>> available for everybody ?
>>>
>>> -Dmitry
>>>
>>>
>>> On 2013-01-17 01:54, Stuart Marks wrote:
>>>> Hi everybody,
>>>>
>>>> I work on the JDK core libraries. It turns out that there is a
>>>> similar library used for testing RMI. Take a look at
>>>> jdk/test/java/rmi/testlibrary. It's been around for quite some time
>>>> already, but I've been doing some maintenance on it recently.
>>>> (Also, it still needs more work.)
>>>>
>>>> Since the RMI library is in the jdk repo and these test utilities are
>>>> in hotspot, it seems like sharing of code might be difficult.
>>>> But we might want to compare notes. Here are some observations and
>>>> comments (though not an actual code review).
>>>>
>>>> * The use of a pair of threads (StreamPumper) to collect output from
>>>> the subprocess is very similar to what RMI does. See the StreamPipe
>>>> class in RMI's test library, which are used by the JavaVM utility
>>>> class. One thing I did do recently was to make sure that the
>>>> subprocess is waited for and the threads are joined. This hadn't been
>>>> done before, leading to race conditions where some subprocess output
>>>> was dropped.
>>>>
>>>> * Since these threads are specifically joined, I'm not sure of the
>>>> utility of making them daemon threads.
>>>>
>>>> * My style in writing test code and test library code is to be quite
>>>> verbose about everything and to check every possible error condition.
>>>> Nothing is worse than seeing a test fail and not having enough
>>>> information in the log to find out why. Actually, I can think of
>>>> something worse: having a test pass erroneously because some error
>>>> condition wasn't checked. (Yes, this has happened.)
>>>>
>>>> * Another style point is that I think it's OK for test library code
>>>> just to throw exceptions out to the caller. My test code usually has
>>>> 'throws IOException' or even 'throws Exception' on most of the
>>>> methods, including main, so having test library code throw checked
>>>> exceptions is reasonable. Jtreg will catch exceptions thrown from
>>>> main and log them. Usually you want tests to fail quickly at the
>>>> point of error. So, I concur with Vitaly's concern about ProcessTools
>>>> handling of IOException and about letting interrupts propagate.
>>>>
>>>> * Actually I have a suggestion from Mandy Chung for RMI's StreamPipe
>>>> threads to catch exceptions and report them back to their "parent"
>>>> JavaVM. This is probably a rare occurrence but it does ensure that
>>>> any errors that occur here won't go unnoticed. See
>>>> JDK-8005827 (
>>>> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=8005827 )
>>>>
>>>> * Regarding jtreg and test library code, it's pretty simple to use.
>>>> In each test that uses the library code, one adds the @library tag to
>>>> declare the library. It's probably a good idea to add an @build tag
>>>> to build each library class that's used in the test. Things usually
>>>> work in the absence of @build tags, because of implicit compilation,
>>>> but it's kind of fragile.
>>>>
>>>> Anyway, I just wanted to share some notes and ideas.
>>>>
>>>> s'marks
>>>>
>>>>
>>>> On 1/16/13 4:34 AM, Christian Törnqvist wrote:
>>>>> Hi everyone,
>>>>>
>>>>> ?
>>>>>
>>>>> This RFE adds a few utility classes to make it a bit easier to write
>>>>> multi-process tests in jtreg, webrev can be found at
>>>>> http://cr.openjdk.java.net/~brutisso/8006413/webrev.00/
>>>>>
>>>>> ?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Christian
>>>>>
>>>
>>> -- Dmitry Samersoff Oracle Java development team, Saint Petersburg,
>>> Russia * Give Rabbit time, and he'll always get the answer
>>>
>>
>> --
>> Dmitry Samersoff
>> Oracle Java development team, Saint Petersburg, Russia
>> * Give Rabbit time, and he'll always get the answer