CODETOOLS-7902290 breaks all JTreg tests which use @requires vm.opt.*

[Jonathan Gibbons]

On 10/12/18 8:15 AM, Volker Simonis wrote:
> On Thu, Oct 11, 2018 at 11:09 PM Igor Ignatyev<igor.ignatyev at oracle.com>  wrote:
>> Hi Volker,
>>
>> vm.opt.* options are set by jtreg (RegressionContext::processVMOptions), and their value is expected to be "null" if a flag hasn't been passed to JDK under tests via -javaoptions or -vmoptions. I don't think there is something to fix (at least not in jtreg).
>>
> Well, if "vm.opt.*" options are indeed only options which are taken
> from the command line (i.e. "-javaoption/-vmoption"), than there's
> definitely something we have to fix! Because, as I described before,
> after CODETOOLS-7902290, all test which check such an option in a
> @requires clause (negatively or positively) will now fail if that
> option is not given explicitly on the command line.
>
> If "vm.opt.*" options are really only the options given on the command
> line (please confirm this and please document it somewhere
> prominently!) I think many tests which use them are broken because
> they simply check such options like this:
>
> @requires vm.opt.TieredCompilation == true
>
> This would be wrong and would have to be replaced by:
>
> @requires vm.opt.TieredCompilation == null | vm.opt.TieredCompilation == true
>
> to also account for the case where -XX:TieredCompilation was not
> explicitly set on the command line. Notice that in in such a case, the
> value of "TieredCompilation" in the VM could be either "true" or
> "false" (depending on the platform and VM build configuration), but it
> wouldn't be possible to test that by using "vm.opt.TieredCompilation".
>
> That said, there are three "vm.opt.final.*" options which are set by VMProps:
>
>          vmOptFinalFlag(map, "ClassUnloading");
>          vmOptFinalFlag(map, "UseCompressedOops");
>          vmOptFinalFlag(map, "EnableJVMCI");
>
> Notice that these options are set to the values of the corresponding
> flags in the agent VM . For tests which run in "othervm" mode (or
> which start new VMs with ProcessBuilder or
> jdk.test.lib.process.ProcessTools), that doesn't necessarily
> corresponds to the values of these flags in the testee VM.
>
> Finally, options specified with "-javaoption/-vmoption" and reflected
> in the corresponding "vm.opt.*" flags, may be over-ridden by command
> line options specified for tests which run in "othervm" mode (so
> checking them with a @requires clause is of limited usefulness as
> well).
>
> All this is quite complex and I think we should*really*  document it
> in a prominent place like the "jtreg: Command Line Options" page [1]
> AND the
> "The JDK Test Framework: Tag Language Specification" page [2].
>
> Regards,
> Volker
>
> [1]http://openjdk.java.net/jtreg/command-help.html
> [2]http://openjdk.java.net/jtreg/tag-spec.html
>

Volker,

I will look  at what can be done to improve the documentation,
especially with regard to documenting the limitations of the
@requires mechanism.

That being said, most of the mechanism is provided outside
of jtreg, by the extraPropDefns  extension mechanism specified
in TEST.ROOT.  It was a deliberate design choice to decouple
these classes from jtreg, so that they can evolve faster and
separately from jtreg promotions, according to the needs of
the test suite. So, without diminishing the need for additional
documentation, I'll note that detailed documentation may not
belong in the jtreg pages you specified.

Finally, you are right to observe the inadequacies of the
vm.opt.* mechanism. IMO, it is better to use custom
properties defined by the extraPropDefns mechanism
that provide the result of analyzing the options, as
compared to checking the options themselves. My
prime example of this, when I was writing the jtreg support,
was the "-server" and "-client" options, which are so-called
supported options on all system, but may be no-ops
on some platforms. Therefore, it is more important to
check the internal settings in the VM than to check the
options given to the VM.

-- Jon