RFR 8059557: Test set for "Validate JVM Command-Line Flag Arguments"

[Christian Tornqvist]

Hi Dmitry,

First of all, the code looks really good. One thing that I noticed is that it seems like the invalid values (out of range) is done for all the options with ranges, this seems redundant to me. It should be enough to test one for each data type instead.

I'll continue to review the code tomorrow :)

Thanks,
Christian

-----Original Message-----
From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Dmitry Dmitriev
Sent: Thursday, May 21, 2015 4:57 PM
To: hotspot-dev at openjdk.java.net; Gerard Ziemski
Subject: Re: RFR 8059557: Test set for "Validate JVM Command-Line Flag Arguments"

Hello all,

Recently I correct several typos, so here a new webrev for tests: 
http://cr.openjdk.java.net/~ddmitriev/8059557/webrev.02/
<http://cr.openjdk.java.net/%7Eddmitriev/8059557/webrev.02/>

Thanks,
Dmitry

On 18.05.2015 18:48, Dmitry Dmitriev wrote:
> Hello all,
>
> Please review test set for verifying functionality implemented by JEP
> 245 "Validate JVM Command-Line Flag Arguments"(JDK-8059557). Review 
> request for this JEP can be found there:
> http://mail.openjdk.java.net/pipermail/hotspot-dev/2015-May/018539.htm
> l
>
> I create 3 tests for verifying options with ranges. The tests mostly 
> rely on common/optionsvalidation/JVMOptionsUtils.java. Class in this 
> file contains functions to get options with ranges as list(by parsing 
> new option "-XX:+PrintFlagsRanges" output), run command line test for 
> list of options and other. The actual test code contained in 
> common/optionsvalidation/JVMOption.java file - testCommandLine(), 
> testDynamic(), testJcmd() and testAttach() methods.
> common/optionsvalidation/IntJVMOption.java and 
> common/optionsvalidation/DoubleJVMOption.java source files contain 
> classes derived from JVMOption class for integer and double JVM 
> options correspondingly.
>
> Here are description of the tests:
> 1)
> hotspot/test/runtime/CommandLine/OptionsValidation/TestOptionsWithRang
> es.java
>
> This test get all options with ranges by parsing output of new option 
> "-XX:+PrintFlagsRanges" and verify these options by starting Java and 
> passing options in command line with valid and invalid values.
> Currently it verifies about 106 options which have ranges.
> Invalid values are values which out-of-range. In test used values 
> "min-1" and "max+1".In this case Java should always exit with code 1 
> and print error message about out-of-range value(with one exception, 
> if option is unsigned and passing negative value, then out-of-range 
> error message is not printed because error occurred earlier).
> Valid values are values in range, e.g. min&max and also several 
> additional values. In this case Java should successfully exit(exit 
> code 0) or exit with error code 1 for other reasons(low memory with 
> certain option value etc.). In any case for values in range Java 
> should not print messages about out of range value.
> In any case Java should not crash.
> This test excluded from JPRT because it takes long time to execute and 
> also fails - some options with value in valid range cause Java to 
> crash(bugs are submitted).
>
> 2)
> hotspot/test/runtime/CommandLine/OptionsValidation/TestOptionsWithRang
> es.java
>
> This test get all writeable options with ranges by parsing output of 
> new option "-XX:+PrintFlagsRanges" and verify these options by 
> dynamically changing it's values to the valid and invalid values. Used
> 3 methods for that: DynamicVMOption isValidValue and isInvalidValue 
> methods, Jcmd and by attach method. Currently 3 writeable options with 
> ranges are verified by this test.
> This test pass in JPRT.
>
> 3) 
> hotspot/test/runtime/CommandLine/OptionsValidation/TestJcmdOutput.java
>
> This test verified output of Jcmd when out-of-range value is set to 
> the writeable option or value violates option constraint. Also this 
> test verify that jcmd not write error message to the target process.
>  This test pass in JPRT.
>
>
> I am not write special tests for constraints for this JEP because 
> there are exist test for that(e.g.
> test/runtime/CompressedOops/ObjectAlignment.java for 
> ObjectAlignmentInBytes or 
> hotspot/test/gc/arguments/TestHeapFreeRatio.java for 
> MinHeapFreeRatio/MaxHeapFreeRatio).
>
> Webrev: http://cr.openjdk.java.net/~ddmitriev/8059557/webrev.00/
> <http://cr.openjdk.java.net/%7Eddmitriev/8059557/webrev.00/>
>
> JEP: https://bugs.openjdk.java.net/browse/JDK-8059557
>
> Thanks,
> Dmitry
>