RFR: 8231296: ZGC: vmTestbase/nsk/jvmti/Allocate/alloc001/ fails

[Daniel D. Daugherty]

On 9/27/19 2:16 AM, Per Liden wrote:
> Hi Igor,
>
> On 9/27/19 3:49 AM, Igor Ignatev wrote:
>> Hi Per,
>>
>>> On Sep 26, 2019, at 2:32 PM, Per Liden <per.liden at oracle.com> wrote:
>>>
>>> Hi Igor,
>>>
>>> I don't think it belongs in the problem list, for two reasons:
>>>
>>> 1) The test doesn't fail because of a bug. It fails because ZGC 
>>> doesn't currently support that use case. In other words, the test 
>>> shouldn't be testing something that isn't supposed to work.
>> Problem lists aren’t only to exclude tests which fail due to bugs, 
>> they can be and are used for use cases you described as well.
>>>
>>> 2) As far as I know, the test in question is part of rt-tiers, where 
>>> it's executed with the default GC (G1). So, the problem is typically 
>>> only visible when doing non-CI testing with ZGC enabled.
>> I agree that necessity to pass extra make arts whenever you are doing 
>> adhoc testing, esp. on a localhost, is cumbersome, but you are 
>> expected to do that anyway as some of tests are known to fail only w/ 
>> zGC and are in ProblemList-zgc.txt.
>>
>> The main advantage of problem list over @requiers is semi-automatic 
>> reminding to re-enable tests when a defect is fixed/feature is 
>> implemented, it also sends a clear message that we plan to make a 
>> test pass.
>>
>> In compiler, we 1st use @requiers to temporary exclude tests from 
>> graal testing, but soon we realized that there are two different 
>> meanings:1) test isn't able to pass w/ graal and will never be able 
>> to and 2) test can’t pass w/ graal now (b/c of test / product bugs 
>> and/or some features aren’t there), but we want to fix product/test 
>> and reenable test. we now try to use @requiers for 1 and problem list 
>> for 2, so the reasons and expectations are clearer, but there are 
>> still some tests which have @requiers which actually should be in 
>> problem list, and it’s expensive to go thru all of them to dig out 
>> which it should be. So I just don’t want you guys to get the same 
>> problems we got.
>
> Ok, thanks. I think I see the problem list more of a temporary measure 
> to avoid noise in the CI pipeline, i.e. when the test should work but 
> doesn't. In cases where a test isn't expected to work, I think 
> @requires is better. However, I do see your point, it's not always a 
> black or white call.

In your code review invite you say this:

> The root problem is that ZGC doesn't properly respect address space 
> limitations (RLIMIT_AS).

To me that says the current behavior is temporary and then you say this:

> This test will be re-enabled again as part of fixing 
> https://bugs.openjdk.java.net/browse/JDK-8231552

and that confirms that the issue is planned to be resolved. I'm not
sure that I agree that 8231552 is an RFE rather than a bug, but that's
not the point here...

To me, both of those sentences would lead to a ProblemListing
and not an @requires. Your call...

Dan


>
> cheers,
> Per
>
>>>
>>> cheers,
>>> Per
>>>
>>>> On 9/26/19 10:58 PM, Igor Ignatyev wrote:
>>>> Hi Per,
>>>> wouldn't it be better to put this test into zgc-specific problem 
>>>> list (test/hotspot/jtreg/ProblemList-zgc.txt)?
>>>> Thanks,
>>>> -- Igor
>>>>>> On Sep 26, 2019, at 1:39 PM, Per Liden <per.liden at oracle.com> wrote:
>>>>>
>>>>> Please review this one-liner to disable 
>>>>> vmTestbase/nsk/jvmti/Allocate/alloc001 when using ZGC. The root 
>>>>> problem is that ZGC doesn't properly respect address space 
>>>>> limitations (RLIMIT_AS). This test will be re-enabled again as 
>>>>> part of fixing https://bugs.openjdk.java.net/browse/JDK-8231552
>>>>>
>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8231296
>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8231296/webrev.0
>>>>>
>>>>> /Per
>>