RFR 8059557: Test set for "Validate JVM Command-Line Flag Arguments"

[Gerard Ziemski]

Looks good. Please consider it reviewed, with small "r".


cheers

On 5/28/2015 8:32 AM, Dmitry Dmitriev wrote:
> Hello all,
>
> Here is a 3 version of the tests taking into account feedback from 
> Christian, David and Gerard.
>
> I limit number of options in TestOptionsWithRanges.java to 15. This 
> include options of different types(intx, uintx etc.) and with 
> different combination of min/max range. 
> TestOptionsWithRangesDynamic.java leaved as is, because amount of 
> manageable numeric options is very small and currently only 3 of them 
> have range. Also, I improve code for double option.
>
> Webrev: http://cr.openjdk.java.net/~ddmitriev/8059557/webrev.03/ 
> <http://cr.openjdk.java.net/%7Eddmitriev/8059557/webrev.03/>
>
> Thanks,
> Dmitry
>
>
> On 21.05.2015 23:57, Dmitry Dmitriev wrote:
>> Hello all,
>>
>> Recently I correct several typos, so here a new webrev for tests: 
>> http://cr.openjdk.java.net/~ddmitriev/8059557/webrev.02/ 
>> <http://cr.openjdk.java.net/%7Eddmitriev/8059557/webrev.02/>
>>
>> Thanks,
>> Dmitry
>>
>> On 18.05.2015 18:48, Dmitry Dmitriev wrote:
>>> Hello all,
>>>
>>> Please review test set for verifying functionality implemented by 
>>> JEP 245 "Validate JVM Command-Line Flag Arguments"(JDK-8059557). 
>>> Review request for this JEP can be found there: 
>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2015-May/018539.html
>>>
>>> I create 3 tests for verifying options with ranges. The tests mostly 
>>> rely on common/optionsvalidation/JVMOptionsUtils.java. Class in this 
>>> file contains functions to get options with ranges as list(by 
>>> parsing new option "-XX:+PrintFlagsRanges" output), run command line 
>>> test for list of options and other. The actual test code contained 
>>> in common/optionsvalidation/JVMOption.java file - testCommandLine(), 
>>> testDynamic(), testJcmd() and testAttach() methods. 
>>> common/optionsvalidation/IntJVMOption.java and 
>>> common/optionsvalidation/DoubleJVMOption.java source files contain 
>>> classes derived from JVMOption class for integer and double JVM 
>>> options correspondingly.
>>>
>>> Here are description of the tests:
>>> 1) 
>>> hotspot/test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java
>>>
>>> This test get all options with ranges by parsing output of new 
>>> option "-XX:+PrintFlagsRanges" and verify these options by starting 
>>> Java and passing options in command line with valid and invalid 
>>> values. Currently it verifies about 106 options which have ranges.
>>> Invalid values are values which out-of-range. In test used values 
>>> "min-1" and "max+1".In this case Java should always exit with code 1 
>>> and print error message about out-of-range value(with one exception, 
>>> if option is unsigned and passing negative value, then out-of-range 
>>> error message is not printed because error occurred earlier).
>>> Valid values are values in range, e.g. min&max and also several 
>>> additional values. In this case Java should successfully exit(exit 
>>> code 0) or exit with error code 1 for other reasons(low memory with 
>>> certain option value etc.). In any case for values in range Java 
>>> should not print messages about out of range value.
>>> In any case Java should not crash.
>>> This test excluded from JPRT because it takes long time to execute 
>>> and also fails - some options with value in valid range cause Java 
>>> to crash(bugs are submitted).
>>>
>>> 2) 
>>> hotspot/test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java
>>>
>>> This test get all writeable options with ranges by parsing output of 
>>> new option "-XX:+PrintFlagsRanges" and verify these options by 
>>> dynamically changing it's values to the valid and invalid values. 
>>> Used 3 methods for that: DynamicVMOption isValidValue and 
>>> isInvalidValue methods, Jcmd and by attach method. Currently 3 
>>> writeable options with ranges are verified by this test.
>>> This test pass in JPRT.
>>>
>>> 3) 
>>> hotspot/test/runtime/CommandLine/OptionsValidation/TestJcmdOutput.java
>>>
>>> This test verified output of Jcmd when out-of-range value is set to 
>>> the writeable option or value violates option constraint. Also this 
>>> test verify that jcmd not write error message to the target process.
>>>  This test pass in JPRT.
>>>
>>>
>>> I am not write special tests for constraints for this JEP because 
>>> there are exist test for that(e.g. 
>>> test/runtime/CompressedOops/ObjectAlignment.java for 
>>> ObjectAlignmentInBytes or 
>>> hotspot/test/gc/arguments/TestHeapFreeRatio.java for 
>>> MinHeapFreeRatio/MaxHeapFreeRatio).
>>>
>>> Webrev: http://cr.openjdk.java.net/~ddmitriev/8059557/webrev.00/ 
>>> <http://cr.openjdk.java.net/%7Eddmitriev/8059557/webrev.00/>
>>>
>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8059557
>>>
>>> Thanks,
>>> Dmitry
>>>
>>
>
>