RFR 8059557: Test set for "Validate JVM Command-Line Flag Arguments"

[Dmitry Dmitriev]

Hello all,

Here is a 3 version of the tests taking into account feedback from 
Christian, David and Gerard.

I limit number of options in TestOptionsWithRanges.java to 15. This 
include options of different types(intx, uintx etc.) and with different 
combination of min/max range. TestOptionsWithRangesDynamic.java leaved 
as is, because amount of manageable numeric options is very small and 
currently only 3 of them have range. Also, I improve code for double 
option.

Webrev: http://cr.openjdk.java.net/~ddmitriev/8059557/webrev.03/ 
<http://cr.openjdk.java.net/%7Eddmitriev/8059557/webrev.03/>

Thanks,
Dmitry


On 21.05.2015 23:57, Dmitry Dmitriev wrote:
> Hello all,
>
> Recently I correct several typos, so here a new webrev for tests: 
> http://cr.openjdk.java.net/~ddmitriev/8059557/webrev.02/ 
> <http://cr.openjdk.java.net/%7Eddmitriev/8059557/webrev.02/>
>
> Thanks,
> Dmitry
>
> On 18.05.2015 18:48, Dmitry Dmitriev wrote:
>> Hello all,
>>
>> Please review test set for verifying functionality implemented by JEP 
>> 245 "Validate JVM Command-Line Flag Arguments"(JDK-8059557). Review 
>> request for this JEP can be found there: 
>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2015-May/018539.html
>>
>> I create 3 tests for verifying options with ranges. The tests mostly 
>> rely on common/optionsvalidation/JVMOptionsUtils.java. Class in this 
>> file contains functions to get options with ranges as list(by parsing 
>> new option "-XX:+PrintFlagsRanges" output), run command line test for 
>> list of options and other. The actual test code contained in 
>> common/optionsvalidation/JVMOption.java file - testCommandLine(), 
>> testDynamic(), testJcmd() and testAttach() methods. 
>> common/optionsvalidation/IntJVMOption.java and 
>> common/optionsvalidation/DoubleJVMOption.java source files contain 
>> classes derived from JVMOption class for integer and double JVM 
>> options correspondingly.
>>
>> Here are description of the tests:
>> 1) 
>> hotspot/test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java
>>
>> This test get all options with ranges by parsing output of new option 
>> "-XX:+PrintFlagsRanges" and verify these options by starting Java and 
>> passing options in command line with valid and invalid values. 
>> Currently it verifies about 106 options which have ranges.
>> Invalid values are values which out-of-range. In test used values 
>> "min-1" and "max+1".In this case Java should always exit with code 1 
>> and print error message about out-of-range value(with one exception, 
>> if option is unsigned and passing negative value, then out-of-range 
>> error message is not printed because error occurred earlier).
>> Valid values are values in range, e.g. min&max and also several 
>> additional values. In this case Java should successfully exit(exit 
>> code 0) or exit with error code 1 for other reasons(low memory with 
>> certain option value etc.). In any case for values in range Java 
>> should not print messages about out of range value.
>> In any case Java should not crash.
>> This test excluded from JPRT because it takes long time to execute 
>> and also fails - some options with value in valid range cause Java to 
>> crash(bugs are submitted).
>>
>> 2) 
>> hotspot/test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java
>>
>> This test get all writeable options with ranges by parsing output of 
>> new option "-XX:+PrintFlagsRanges" and verify these options by 
>> dynamically changing it's values to the valid and invalid values. 
>> Used 3 methods for that: DynamicVMOption isValidValue and 
>> isInvalidValue methods, Jcmd and by attach method. Currently 3 
>> writeable options with ranges are verified by this test.
>> This test pass in JPRT.
>>
>> 3) 
>> hotspot/test/runtime/CommandLine/OptionsValidation/TestJcmdOutput.java
>>
>> This test verified output of Jcmd when out-of-range value is set to 
>> the writeable option or value violates option constraint. Also this 
>> test verify that jcmd not write error message to the target process.
>>  This test pass in JPRT.
>>
>>
>> I am not write special tests for constraints for this JEP because 
>> there are exist test for that(e.g. 
>> test/runtime/CompressedOops/ObjectAlignment.java for 
>> ObjectAlignmentInBytes or 
>> hotspot/test/gc/arguments/TestHeapFreeRatio.java for 
>> MinHeapFreeRatio/MaxHeapFreeRatio).
>>
>> Webrev: http://cr.openjdk.java.net/~ddmitriev/8059557/webrev.00/ 
>> <http://cr.openjdk.java.net/%7Eddmitriev/8059557/webrev.00/>
>>
>> JEP: https://bugs.openjdk.java.net/browse/JDK-8059557
>>
>> Thanks,
>> Dmitry
>>
>