RFR 8252249: nsk/stress/stack/stack016.java fails with "Error: TEST_BUG: trickyRecursion() must throw an error anyway!"

Igor Ignatyev igor.ignatyev at oracle.com
Wed Sep 2 22:16:20 UTC 2020


Hi David,

> On Sep 2, 2020, at 2:58 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> Hi Igor,
> 
> On 3/09/2020 6:33 am, Igor Ignatyev wrote:
>> Hi Harold,
>> currently, the configurations which a test is run defined by test execution configuration (which is tiers/tasks-definitions in Oracle's case, and I'd guess others their own configuration matrixes), not a test (tests are, however, responsible for filtering out incompatible/meaningless configurations). this particular test is included in runs w/ tens different flag combinations, w/ around a half of them having -Xcomp; and `vm.compMode != "Xcomp"` effectively removes the test from all these runs.
> 
> Note that the test already doesn't run in those cases as it already has the `vm.compMode != "Xcomp"`.
 
right, form that perspective the patch has actually increased the number of Xcomp configurations. 
> But I agree that your suggestion would expose the test to more Xcomp testing configurations, which is potentially a good thing. The only concern is that we don't yet know how the test will behave in those different Xcomp configurations.

that's a valid concern, yet it's a concern w/ any changes, and there are two ways to go about it:
 - run the test once time in each (Xcomp) configurations
 - run the test N times in each (Xcomp) configurations (w/ N being big enough to provide statistically significant amount of data to show stability of the test)

in both cases, if the results are good, you push and observer results for regular testing and react accordingly; if the results aren't so good, you fix and reiterate. from my experience, running the test huge number of times in each "regularly" used configuration is a waste of machine time, as you will never get 100% certainly that the test is stable, and while you are testing, the product will get changed, so all the data you collected is, strictly speaking, irrelevant.

anyhow, I see that 8252249 has already been integrated, so I've filed JDK-8252723 to make this test executed in Xcomp configs, and we can discuss the best approach to test the test there.

Thanks,
-- Igor

> 
> David
> 
>> Thanks,
>> -- Igor
>>> On Sep 2, 2020, at 1:21 PM, Harold Seigel <harold.seigel at oracle.com <mailto:harold.seigel at oracle.com>> wrote:
>>> 
>>> Hi Igor,
>>> 
>>> I think that the test needs to only run twice, once with -Xcomp and once with -Xint.  It does not need to run with multiple -Xcomp configurations.  It needs only to run with consistent stack sizes, so that stack consumption doesn't vary because some threads run an interpreted method and others run a compiled method.
>>> 
>>> My understanding is that "vm.compMode != "Xcomp" will cause the test to be run without -Xcomp.  The test will then run using its internal -Xint flag and then its -Xcomp flags.  Which I think is what we want.
>>> 
>>> Thanks, Harold
>>> 
>>> On 9/2/2020 10:45 AM, Igor Ignatyev wrote:
>>>> Hi Harold,
>>>> 
>>>> vm.compMode != "Xcomp" will exclude all combinations which has Xcomp, e.g. '-Xcomp -XX:-TieredCompilation', so it would be waste iff all other flags from Xcomp configurations are also used w/o Xcomp, which I don't think is generally true.
>>>> 
>>>> given vm flags from jtreg actions are appended to external flags, running this test w/ '-Xcomp -XX:-TieredCompilation' (assuming there is no @requires) will actually result in two different (and one might argue expected) runs:
>>>>  -Xcomp -XX:-TieredCompilation -Xint -Xss448K nsk.stress.stack.stack016 -eager
>>>>  -Xcomp -XX:-TieredCompilation -Xcomp -Xss448K nsk.stress.stack.stack016 -eager
>>>> 
>>>> thus I don't think '@requires vm.compMode' is needed here.
>>>> 
>>>> if you think that '-Xint' run should be excluded when one specifies -Xcomp externally, I'd suggest splitting the test into two
>>>> 
>>>> /*
>>>>  * @test id=Xint
>>>>  * @requires vm.compMode == null | vm.compMode == "Xint"
>>>>  * @requires (vm.opt.DeoptimizeALot != true)
>>>>  * @library /vmTestbase
>>>>  * @build nsk.share.Terminator
>>>>  * @run main/othervm/timeout=900 -Xint -Xss448K nsk.stress.stack.stack016 -eager
>>>>  */
>>>> 
>>>> /*
>>>>  * @test id=Xcomp
>>>>  * @requires vm.compMode == null | vm.compMode == "Xcomp"
>>>>  * @requires (vm.opt.DeoptimizeALot != true)
>>>>  * @library /vmTestbase
>>>>  * @build nsk.share.Terminator
>>>>  * @run main/othervm/timeout=900 -Xcomp -Xss448K nsk.stress.stack.stack016 -eager
>>>>  */
>>>> 
>>>> in this case Xint run will be run only if there is no Xint/Xmixed/Xbatch/Xcomp specified externally or Xint is specified, and similarly for Xcomp.
>>>> 
>>>> Thanks,
>>>> -- Igor
>>>> 
>>>> 
>>>> 
>>>>> On Sep 1, 2020, at 11:26 PM, David Holmes <david.holmes at oracle.com <mailto:david.holmes at oracle.com>> wrote:
>>>>> 
>>>>> I would suggest that you keep the required condition
>>>>> 
>>>>> vm.compMode != "Xcomp"
>>>>> 
>>>>> to minimise the runs of this test. When executed as part of an Xcomp run both @run's will behave exactly the same way (Xcomp) - and that is the same as the second @run in a non-Xcomp run. So no point running the Xcomp version three times.
>>>> 



More information about the hotspot-runtime-dev mailing list