RFR (L): 8024319: Add regression tests on documented GC-related -XX:+Print* options

[Dmitry Fazunenko]

Hi Bengt!

Thanks for looking at the tests and sharing your thoughts!

Yes, it was considered to use ProcessTool but we decided to introduce 
AbstractPringGCTest class.
Our thoughts:
- log generation and log analysis are two different tasks, and they 
should logically separated:
     * starting GCTask with various options  (one line @run main/othervm 
... GCTask)
     * log analysis (the java code within sources)
   this increases readability
- it's possible to provide several checkers for the same log
     @run main/othervm ... GCTask
     @run main TestCheck1
     @run main TestCheck2
     ...
     @run main TestCheck3
   or provide the same check for several logs (not currently implemented)
     @run main/othervm ... -loggc:log1 GCTask
     @run main/othervm ... -loggc:log2 GCTask
     @run main/othervm ... -loggc:log3 GCTask
     ...
     @run main TestCheck log1 log2 log3

- writing log to a dedicated file will guarantee, that program output 
and the GC log will not be mixed up.
   (not valid argument for that particular case, bun in general that's true)

- using @run main/othervm will allow to *not ignore* external options 
(jtreg -vmoptions:...), that makes such tests applicable for wider range 
of configurations and check, that for example -Xcomp doesn't turn off 
PrintGCDetails...


Regarding AbstractPringGCTes: it doesn't duplicate any functionality, it 
just reads content of a text file.

Regarding ProcessTools. Yes, it's possible to develop tests with this 
library. This library itself is very good and powerful. But I personally 
would not recommend using it for test development because it duplicates 
harness functionality. Unfortunately, jtreg currently doesn't provide 
all functionality required for VM testing and we have to use 
ProcessTools as workaround.
And people already got used to ProcessTools and like this style. But in 
long term, there will be the same problem with support of such tests, as 
we experience now with tests written in shell. Indeed, using 
ProcessTools is like using java for shell scripting.

Returning to Denis' tests. They intentionally do not use ProcessTools. 
They could be used as example demonstrating an alternative approach.

Thanks,
Dima



On 22.01.2015 10:26, Bengt Rutisson wrote:
>
> On 2015-01-22 08:20, Bengt Rutisson wrote:
>>
>> Hi Denis,
>>
>> Not a full review, but I have a question.
>>
>> It seems like the AbstractPrintGCTest is kind of duplicating what 
>> ProcessTools already does. Have you considered using 
>> ProcessTools.createJavaProcessBuilder(..) instead of the @run 
>> commands to automatically get the process control and log support 
>> instead of introducing the AbstractPrintGCTest class?
>
> Here's an example of how I mean that you can use 
> ProcessTools.createJavaProcessBuilder() instead:
>
> http://hg.openjdk.java.net/jdk9/hs-gc/hotspot/file/e6a0cfbfdc9a/test/gc/g1/TestGCLogMessages.java 
>
>
> Bengt
>
>>
>> Thanks,
>> Bengt
>>
>> On 2015-01-20 16:49, denis kononenko wrote:
>>> Hi All,
>>>
>>> Could you please review new tests on GC-related -XX:Print options.
>>>
>>> Webrev link: 
>>> http://cr.openjdk.java.net/~eistepan/dkononen/8024319/webrev.00/
>>> Bug id: https://bugs.openjdk.java.net/browse/JDK-8024319
>>> Testing: automated
>>> Description:
>>>
>>> There is a group of new tests to test the following GC options:
>>>
>>> -XX:+-PrintAdaptiveSizePolicy
>>> -XX:+-PrintGCApplicationConcurrentTime
>>> -XX:+-PrintGCApplicationStoppedTime
>>> -XX:+-PrintGCDateStamps
>>> -XX:+-PrintGCDetails
>>> -XX:+-PrintGC
>>> -XX:+-PrintGCTimeStamps
>>> -XX:+-PrintTenuringDistribution
>>>
>>> Each of the tested options has a pair of corresponding tests, one is 
>>> for testing an enabled option and another for disabled. The tests 
>>> are simple parsers of GC's logger output and looking for a specific 
>>> marker corresponding to the tested option. The output is provided by 
>>> another process which is implemented in GCTask.java. It's necessary 
>>> because we have to guarantee that GC's log has been completely 
>>> written and committed to the filesystem before we start analyzing 
>>> it. The most obvious solution is to politely finish the writing 
>>> process. Thus the every test spawns an auxiliary process which 
>>> produces GC's log file, waits for its finish, loads the output and 
>>> then performs actual testing. These steps are implemented with 
>>> jtreg's annotations and a helper class which can be found 
>>> AbstractPrintGCTest.java. This class encapsulates reading GC's log 
>>> output from the log file and provides that output to the tests.
>>>
>>> To get GC's logger working GCTask forces the garbage collecting 
>>> process. It attempts to consume all memory available to the young 
>>> generation by creating a lot of unreferenced objects. Sooner or 
>>> later the garbage collector shall be invoked. In favor of 
>>> performance the task is implemented to be ran with a small memory 
>>> size less or equal to 128 megabytes. This is excplicitly specified 
>>> with -Xmx JVM's option in jtreg's annotations.
>>>
>>> Please note that some options work for specific GCs only. To prevent 
>>> them from being executed against wrong GC jtreg's annotations and 
>>> groups are used.
>>>
>>> Thank you,
>>> Denis.
>>>
>>>
>>>
>>
>