RFR: 8224112: [TESTBUG] java/util/Base64/TestEncodingDecodingLength.java should be skipped with insufficient memory

[Jie Fu]

Okay, I see.

Thank you very much, Aleksey.

On 2019/5/18 下午5:00, Aleksey Shipilev wrote:
> On 5/18/19 10:10 AM, Jie Fu wrote:
>> If the memory isn't enough, the test won't run (filtered out by the "requires" tag) instead of
>> being skipped in the previous patch. It fixes the false sense of passing problem pointed out by
>> Aleksey. Could you please review it and give me some advice?
> Still against it, sorry. You are asking to skip the test based on the transient state of the system,
> that is not related to the product at all. Suppose the test fails due to a product bug. A developer
> can run the tests on busy machine, get the test skipped because there is no free memory (it is
> hoarded by other developers), and come thinking that product is not broken. Checking the "free
> memory" does not resolve the false sense of passing tests in that scenario.
>
> Once again, having the infrastructure-induced (false) test failure is better than having the test
> that allows bugs to creep in. Get more memory on a test machine (16 GB on a shared test machine is
> heavily problematic!), or figure out the ways you schedule tests -- maybe with the help of CI that
> serializes test runs (that is what I do), or skip the test failure manually when analyzing the
> results (that is what I do as well).
>
> -Aleksey
>