RFR 8059557: Test set for "Validate JVM Command-Line Flag Arguments"

[Dmitry Dmitriev]

Hi Christian,

Thank you for reviewing the code! I will limit number of options to 
test. But I think instead of one option for each type I will add several 
options for each type with different combination of min/max range(e.g. 
max range = maximum number for type, min = small or min = minimum number 
for type, max = not huge and so on). I expect to have about 15 options 
in this case.

Thanks,
Dmitry

On 27.05.2015 0:55, Christian Tornqvist wrote:
> Hi Dmitry,
>
> First of all, the code looks really good. One thing that I noticed is that it seems like the invalid values (out of range) is done for all the options with ranges, this seems redundant to me. It should be enough to test one for each data type instead.
>
> I'll continue to review the code tomorrow :)
>
> Thanks,
> Christian
>
> -----Original Message-----
> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Dmitry Dmitriev
> Sent: Thursday, May 21, 2015 4:57 PM
> To: hotspot-dev at openjdk.java.net; Gerard Ziemski
> Subject: Re: RFR 8059557: Test set for "Validate JVM Command-Line Flag Arguments"
>
> Hello all,
>
> Recently I correct several typos, so here a new webrev for tests:
> http://cr.openjdk.java.net/~ddmitriev/8059557/webrev.02/
> <http://cr.openjdk.java.net/%7Eddmitriev/8059557/webrev.02/>
>
> Thanks,
> Dmitry
>
> On 18.05.2015 18:48, Dmitry Dmitriev wrote:
>> Hello all,
>>
>> Please review test set for verifying functionality implemented by JEP
>> 245 "Validate JVM Command-Line Flag Arguments"(JDK-8059557). Review
>> request for this JEP can be found there:
>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2015-May/018539.htm
>> l
>>
>> I create 3 tests for verifying options with ranges. The tests mostly
>> rely on common/optionsvalidation/JVMOptionsUtils.java. Class in this
>> file contains functions to get options with ranges as list(by parsing
>> new option "-XX:+PrintFlagsRanges" output), run command line test for
>> list of options and other. The actual test code contained in
>> common/optionsvalidation/JVMOption.java file - testCommandLine(),
>> testDynamic(), testJcmd() and testAttach() methods.
>> common/optionsvalidation/IntJVMOption.java and
>> common/optionsvalidation/DoubleJVMOption.java source files contain
>> classes derived from JVMOption class for integer and double JVM
>> options correspondingly.
>>
>> Here are description of the tests:
>> 1)
>> hotspot/test/runtime/CommandLine/OptionsValidation/TestOptionsWithRang
>> es.java
>>
>> This test get all options with ranges by parsing output of new option
>> "-XX:+PrintFlagsRanges" and verify these options by starting Java and
>> passing options in command line with valid and invalid values.
>> Currently it verifies about 106 options which have ranges.
>> Invalid values are values which out-of-range. In test used values
>> "min-1" and "max+1".In this case Java should always exit with code 1
>> and print error message about out-of-range value(with one exception,
>> if option is unsigned and passing negative value, then out-of-range
>> error message is not printed because error occurred earlier).
>> Valid values are values in range, e.g. min&max and also several
>> additional values. In this case Java should successfully exit(exit
>> code 0) or exit with error code 1 for other reasons(low memory with
>> certain option value etc.). In any case for values in range Java
>> should not print messages about out of range value.
>> In any case Java should not crash.
>> This test excluded from JPRT because it takes long time to execute and
>> also fails - some options with value in valid range cause Java to
>> crash(bugs are submitted).
>>
>> 2)
>> hotspot/test/runtime/CommandLine/OptionsValidation/TestOptionsWithRang
>> es.java
>>
>> This test get all writeable options with ranges by parsing output of
>> new option "-XX:+PrintFlagsRanges" and verify these options by
>> dynamically changing it's values to the valid and invalid values. Used
>> 3 methods for that: DynamicVMOption isValidValue and isInvalidValue
>> methods, Jcmd and by attach method. Currently 3 writeable options with
>> ranges are verified by this test.
>> This test pass in JPRT.
>>
>> 3)
>> hotspot/test/runtime/CommandLine/OptionsValidation/TestJcmdOutput.java
>>
>> This test verified output of Jcmd when out-of-range value is set to
>> the writeable option or value violates option constraint. Also this
>> test verify that jcmd not write error message to the target process.
>>   This test pass in JPRT.
>>
>>
>> I am not write special tests for constraints for this JEP because
>> there are exist test for that(e.g.
>> test/runtime/CompressedOops/ObjectAlignment.java for
>> ObjectAlignmentInBytes or
>> hotspot/test/gc/arguments/TestHeapFreeRatio.java for
>> MinHeapFreeRatio/MaxHeapFreeRatio).
>>
>> Webrev: http://cr.openjdk.java.net/~ddmitriev/8059557/webrev.00/
>> <http://cr.openjdk.java.net/%7Eddmitriev/8059557/webrev.00/>
>>
>> JEP: https://bugs.openjdk.java.net/browse/JDK-8059557
>>
>> Thanks,
>> Dmitry
>>
>