We need to add blocking methods to CompletionStage!

[Viktor Klang]

On Sep 27, 2016 01:18, "Martin Buchholz" <martinrb at google.com> wrote:
>
>
>
> On Mon, Sep 26, 2016 at 7:55 AM, Viktor Klang <viktor.klang at gmail.com>
wrote:
>>>
>>>
>>>
>>> Test methods,
>>
>>
>> Yeah, I thought so as well, but it turns out that when you have tons of
async tests, not being able to start new tests until either that timeout or
result makes for a very slow test suite, so that's why most serious test
frameworks are growing support for dealing with async code. Then all you
want to be able to limit is work-in-progress (sloppily called parallelism)
>
>
> I don't see any support in junit or testng for multi-threaded tests.
 jtreg uses the strategy of having a set of reusable JVM processes, each of
which is running only one test at a time, which provides "pretty good"
isolation, and seems to work well.

It's less about multithreading per-se and more about async/non-blocking.

Here's one example of a fwk which supports it:
http://www.scalatest.org/user_guide/async_testing

>
>>>
>>> main methods
>>
>>
>> That's a common scenario, but one that can be solved by having
non-daemonic pooled worker threads.
>
>
> Do you have docs for such a thread pool?

You mean one which has a ThreadFactory which enables setDaemonic(false)?

>
>>>
>>> and unix process reapers are all examples where it's reasonable to
block forever.
>>
>>
>> What about waitpid() + WNOHANG?
>
>
> Are you suggesting periodic polling is better than blocking?

The botion of "better" requires some context to measure against:

>From a liveness, responsiveness and scalability PoV, I'd say, barring
selector support (which is arguably also polling), it's the next best
thing. Since it allows you to monitor N number of external processes with a
low constant factor.

>
>>>>
>>>> PPPPS: "I think it's unreasonable to not provide this for users
(especially when we can do so more efficiently)." <- If efficiency is
desired then blocking is definitely not the right solution.
>>>
>>>
>>> What about efficiently providing isComplete?
>>
>>
>> In my experience isComplete is virtually useless without being able to
extract the value, in which case you may as well introduce a non-blocking
`Optional<T> poll()`
>
>
> Do you support adding the Polling methods from
> http://www.scala-lang.org/api/2.12.0-RC1/scala/concurrent/Future.html
> to CompletionStage, i.e. isDone and getNow?

After thinking about it, I'd say 'No'. Since then people would ask to add
more polling options: isFailed etc.
Having been a part of creating s.c.Future I'd say adding the polling
methods (barring possibly 'value: Option[Try[T]]') has been proven to add
little in terms of value while adding a much larger API surface area.
Since Java lacks value-level try-catch (scala.util.Try) and Java lacks
disjoint union types, the signature of 'getNow' would have to throw, which
is generally a bad thing.

So I think a PollableCompletionStage would be a better option, where the
broader surface area of these kinds of ops can be contained.

Adding a CompletionStage constructor to CompletableFuture would make
conversions simple.

Reiterating Dougs sentiment: there is no API which will appease everyone,
and if everything always should be added, not only does every API become a
kitchen sink, but also all implementations thereof will be needlessly
complex, slow and hard to maintain.

>
>>>
>>> The result may already be available without actually blocking.  It may
even be known to be available immediately.  One would like to get the value
without additional allocation.
>>
>>
>> I've seen that use-case :), and it tends to either be a situation where
the value is available by pure luck (or…scheduling artifacts) or when one
is keeping CompletionStages where strict values could be kept instead
(think rebinding a value on completion).
>>
>> Reading what your'e writing, may I dare propose that what you're after
is something along the lines of a: PollableCompletionStage which sits in
between CompletionStage and CompletableFuture?
>
>
> I've been waffling!  Right now, I'm leaning towards having Java APIs
return fully mutable CompletableFutures as Benjamin and Pavel suggested
upthread.  Why?  Because a completion stage can be thought of as all of:
> - a consumer of an upstream value

Not this, CompletionStage doesn't specify how its value comes to be.

> - a subscription to the event that makes the upstream value available,
and

Never this, since CompletionStage is not cancellable.

> - the producer of a possible value for downstream consumers.

This is exactly what it is.

> Cancellation could be interpreted as a request to unsubscribe from
upstream producer or a notification to downstream consumers that no value
will be forthcoming, or both.

CompletionStage is not cancellable so this concern does not exist.

>
> Here's a problem with jdk9 minimalCompletionStage: even if you are happy
with the minimality of the source stage (perhaps it's shared) you might
want downstream stages to be mutable, but the methods such as thenApply
also create minimal completion stages.  If you want to convert to a mutable
future, you can try calling toCompletableFuture, but there's no guarantee
that will work, even if it doesn't throw UOE.  You can hand-construct a
CompletableFuture which is then completed in a Runnable via thenRun, but
then this is opaque to the CompletableFuture implementation, and
implementation features such as stack overflow prevention will be
ineffective.

This is what I mean by a missing abstraction: There is no API to define a
workflow which would be cancellable.
Fortunately the basis (an SPI) of such thing will be available: j.u.c.Flow

>
> So right now I'm OK with e.g. Process#onExit simply returning
CompletableFuture.

I would not name it onExit since that signals, to me, a callback, and in
which case it ought to take a callback as a parameter.

Also, what does cancellation of onExit mean?

Cheers,
√