RFR(S): 8151707: Mark hotspot tests which ignore external VM flags

[Dmitry Fazunenko]

Hi Christian.

Thank you for the sharing your thoughts!

I agree with you, that madness rotation of the flags is not an optimal 
way of testing and ideally for each configuration (a combination of 
flags) we need to run a certain set of tests.

On the other hand, at the moment you develop a test, you don't know 
which configurations will have the VM in a year.

My position:
- tests should not ignore external flag by default
- if a test does ignore, there should be a reason
- ability to run test a test in all possible configurations, doesn't 
mean the test will be run in all that configs

So, I believe we need this change to know which tests do not accept 
external options.
It doesn't contradict with your idea of more optimal way to select tests 
to check a configuration.

Thanks,
Dima


On 17.03.2016 18:42, Christian Tornqvist wrote:
> Hi Dima,
>
> I don't think this is the right way to go, we need to stop with the flag rotation madness and this is another step in trying to preserve that. Tests need to explicitly define how they're supposed to run, this is not getting us any nearer that. I don't think we should make this change.
>
> Thanks,
> Christian
>
> -----Original Message-----
> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Dmitry Fazunenko
> Sent: Thursday, March 17, 2016 10:51 AM
> To: hotspot-dev at openjdk.java.net
> Subject: RFR(S): 8151707: Mark hotspot tests which ignore external VM flags
>
> Hello,
>
> Could you review a simple fix with introduces a new jtreg keyword:
> "noflags".
> The purpose of this keyword is to mark those tests which ignore VM flags given outside (on the command line).
> It will allow to not run such tests many times in different VM configurations as we do every night, but run only once.
>
> Note: this fix will not affect execution, so the tests labeled as "noflags" will not be automatically filtered out in case of external VM flags are given. It will just allow to not execute such tests:
>     # jtreg  -k:'!noflags' -vmflags:'-XX:+UseG1GC -Xint' test/
>
> Note2: All the tests modified tests are manually reviewed to be sure that they indeed ignore external VM flags.
>
> https://bugs.openjdk.java.net/browse/JDK-8151707
> http://cr.openjdk.java.net/~dfazunen/8151707/webrev.01/
>
> Thanks,
> Dima
>