RFR: JDK-8304557: java/util/concurrent/CompletableFuture/CompletableFutureOrTimeoutExceptionallyTest.java times out

[Pavel Rappo]

On Wed, 22 Mar 2023 11:21:14 GMT, Viktor Klang <duke at openjdk.org> wrote:

>> Hello Viktor,
>> 
>>> @jaikiran Having a long timeout doesn't seem like a problem given that it just needs enough time to run through the iterations (i.e. that's the max duration of the test before giving up).
>> 
>> Having a extremely long timeout of T1 secods for a test which one can (reasonably) expect to finish in (T1 - X) seconds can mean that when/if it genuinely times out, then it holds on to the limited shared resources (like the host machine) for those X seconds longer instead of potentially letting other tests run during that period. So I think it's always better to have a reasonable timeout instead of an extremely large one - a value past which you know that if the test is still running then it's surely be a sign that it should no longer continue to run.
>> Typically the timeout for such tests is decided by running the test against the hosts/environment where it failed and gathering data to see how long it usually takes to finish successfully on those and then adding some extra seconds to it.
>
> @jaikiran The downside is that it is hard to judge how long it will take for machine X to execute Y instructions. :/

1. Is there a way to mark this test as skipped if the timeout elapses? I suggest we don't mark it as skipped based on the build or any other configuration, but actaully run it and see if it fails or succeeds. The rationale is this: if the test is still running when the time is out, it means that the test is inconclusive.
2. Can we reduce memory configuration to conserve system resources? Say, set `-Xmx` to 96 or even 64 megabytes. Or is so little / unusual that we'll risk getting other issues on modern JVMs?
3. I don't see why we need to reduce the number of iterations. What we should do instead is compute it, otherwise that number, whatever it is, looks magic. There needs to be some rationale behing that number, even if that rationale is not accurate. Say, we can reasonably assume that overhead for one `CompletableFuture` is at least one byte. So, to fill up `N` megabyte(s) of heap we would need `N * 1_048_576` iterations.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/13116#discussion_r1144777727