RFR 8059557: Test set for "Validate JVM Command-Line Flag Arguments"

[Dmitry Dmitriev]

Hello all,

Recently I correct several typos, so here a new webrev for tests: 
http://cr.openjdk.java.net/~ddmitriev/8059557/webrev.02/ 
<http://cr.openjdk.java.net/%7Eddmitriev/8059557/webrev.02/>

Thanks,
Dmitry

On 18.05.2015 18:48, Dmitry Dmitriev wrote:
> Hello all,
>
> Please review test set for verifying functionality implemented by JEP 
> 245 "Validate JVM Command-Line Flag Arguments"(JDK-8059557). Review 
> request for this JEP can be found there: 
> http://mail.openjdk.java.net/pipermail/hotspot-dev/2015-May/018539.html
>
> I create 3 tests for verifying options with ranges. The tests mostly 
> rely on common/optionsvalidation/JVMOptionsUtils.java. Class in this 
> file contains functions to get options with ranges as list(by parsing 
> new option "-XX:+PrintFlagsRanges" output), run command line test for 
> list of options and other. The actual test code contained in 
> common/optionsvalidation/JVMOption.java file - testCommandLine(), 
> testDynamic(), testJcmd() and testAttach() methods. 
> common/optionsvalidation/IntJVMOption.java and 
> common/optionsvalidation/DoubleJVMOption.java source files contain 
> classes derived from JVMOption class for integer and double JVM 
> options correspondingly.
>
> Here are description of the tests:
> 1) 
> hotspot/test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java
>
> This test get all options with ranges by parsing output of new option 
> "-XX:+PrintFlagsRanges" and verify these options by starting Java and 
> passing options in command line with valid and invalid values. 
> Currently it verifies about 106 options which have ranges.
> Invalid values are values which out-of-range. In test used values 
> "min-1" and "max+1".In this case Java should always exit with code 1 
> and print error message about out-of-range value(with one exception, 
> if option is unsigned and passing negative value, then out-of-range 
> error message is not printed because error occurred earlier).
> Valid values are values in range, e.g. min&max and also several 
> additional values. In this case Java should successfully exit(exit 
> code 0) or exit with error code 1 for other reasons(low memory with 
> certain option value etc.). In any case for values in range Java 
> should not print messages about out of range value.
> In any case Java should not crash.
> This test excluded from JPRT because it takes long time to execute and 
> also fails - some options with value in valid range cause Java to 
> crash(bugs are submitted).
>
> 2) 
> hotspot/test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java
>
> This test get all writeable options with ranges by parsing output of 
> new option "-XX:+PrintFlagsRanges" and verify these options by 
> dynamically changing it's values to the valid and invalid values. Used 
> 3 methods for that: DynamicVMOption isValidValue and isInvalidValue 
> methods, Jcmd and by attach method. Currently 3 writeable options with 
> ranges are verified by this test.
> This test pass in JPRT.
>
> 3) hotspot/test/runtime/CommandLine/OptionsValidation/TestJcmdOutput.java
>
> This test verified output of Jcmd when out-of-range value is set to 
> the writeable option or value violates option constraint. Also this 
> test verify that jcmd not write error message to the target process.
>  This test pass in JPRT.
>
>
> I am not write special tests for constraints for this JEP because 
> there are exist test for that(e.g. 
> test/runtime/CompressedOops/ObjectAlignment.java for 
> ObjectAlignmentInBytes or 
> hotspot/test/gc/arguments/TestHeapFreeRatio.java for 
> MinHeapFreeRatio/MaxHeapFreeRatio).
>
> Webrev: http://cr.openjdk.java.net/~ddmitriev/8059557/webrev.00/ 
> <http://cr.openjdk.java.net/%7Eddmitriev/8059557/webrev.00/>
>
> JEP: https://bugs.openjdk.java.net/browse/JDK-8059557
>
> Thanks,
> Dmitry
>