RFR(XS): 8215042: Move serviceability/sa tests from tier1 to tier3.

[Jini George]

Hi Leonid,

Looks like all the SA failures here are all due to 
https://bugs.openjdk.java.net/browse/JDK-8202884. Do let me know if I am 
mistaken. We will work on fixing that issue faster.

Thanks,
Jini.

On 12/10/2018 12:51 PM, Leonid Mesnik wrote:
> David, Jini
> 
> I understand your concerns. But the original idea of tiered testing is 
> that tier1 failures are treated as urgent issues and to resolve. [1]
> 
> Here is list of test failures for 1000 runs of tier1 tests in Mach5. (I 
> am not able to provide a link here) Please note that all SA tests are 
> excluded on Solaris and MacosX already.
> 
> 1 compiler/aot/calls/fromAot/AotInvokeSpecial2AotTest.java
> 2 serviceability/sa/ClhsdbFindPC.java
> 3 serviceability/sa/TestPrintMdo.java
> 4 serviceability/sa/ClhsdbJstack.java
> 5 serviceability/sa/ClhsdbJdis.java
> 6 compiler/c2/Test8004741.java
> 7 runtime/handshake/HandshakeWalkSuspendExitTest.java
> 8 runtime/handshake/HandshakeWalkSuspendExitTest.java
> 9 compiler/aot/calls/fromAot/AotInvokeVirtual2AotTest.java
> 10 runtime/handshake/HandshakeWalkExitTest.java
> 11 runtime/handshake/HandshakeWalkSuspendExitTest.java
> 12 serviceability/sa/ClhsdbRegionDetailsScanOopsForG1.java
> 13 serviceability/sa/ClhsdbRegionDetailsScanOopsForG1.java
> 14 compiler/aot/calls/fromAot/AotInvokeVirtual2AotTest.java
> 
> The failures in of 'runtime/handshake/' are relatively caused by 
> https://bugs.openjdk.java.net/browse/JDK-8214174 but should be also 
> fixed/excluded. SA tests are also unstable and there are no plans to fix 
> them soon.
> So it means that we are going to have tier1 tests unstable for a long time.
> 
> The possible way to make tier1 more stable would be to run only some 
> very basic sanity SA tests in tier1.  Might be to develop new sanity 
> test which have some failover for existing SA bugs.
> 
> Leonid
> 
> [1] http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-March/001991.html
> 
>> On Dec 9, 2018, at 7:19 PM, Jini George <jini.george at oracle.com 
>> <mailto:jini.george at oracle.com>> wrote:
>>
>> Hi Leonid,
>>
>> I agree with David. I am also concerned about us not detecting SA 
>> breakages (which could happen along with hotspot changes) soon enough. 
>> (Which was the primary reason to get these tests in).
>>
>> Thank you,
>> Jini.
>>
>> On 12/8/2018 4:48 PM, David Holmes wrote:
>>> Hi Leonid,
>>> My concern here, if we care about keeping the SA operational, is that 
>>> in tier3 these tests will not be covered by the jdk/submit testing 
>>> process.
>>> David
>>> On 8/12/2018 3:53 pm, Leonid Mesnik wrote:
>>>> Hi
>>>>
>>>> Could you please review following fix which moves SA tests from 
>>>> tier1 to tier3. There are some bugs which cause intermittent 
>>>> failures of any test. SA tests fail intermittently are not stable 
>>>> enough for tier1.
>>>> However failures are not very frequent. Also I don't think that 
>>>> putting all test in Problemlist.txt is very good idea because it 
>>>> left SA without any testing at all.
>>>> So now all SA tests which are included in hotspot_tier3_runtime group.
>>>>
>>>> webrev: http://cr.openjdk.java.net/~lmesnik/8215042/webrev.00/
>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8215042
>>>>
>>>> Leonid
>