j.u.s.Stream and Fiber

[Alan Bateman]

On 21/09/2019 09:02, Arkadiusz Gasiński wrote:
> I actually meant (a) when I started this thread, but I think (b) is 
> the next question to ask if we (I mean you :)) want to ever consider 
> running parallel streams in fibers.
>
> The use case that made me start this thread was: what would happen if 
> I start processing a large number of files in a parallel stream in the 
> context of a Fiber? I've finally found some time to check it and now I 
> know that (at least in the current prototype) some processing will be 
> done in the context of the fiber that triggered terminal operation on 
> the pipeline, but most work will actually be performed 
> by ForkJoinPool.commonPool workers. I have to admit that I was a bit 
> surprised by this - not saying this is good or bad, but for some weird 
> reason was expecting all processing to be done in the context of fiber(s).
>
This is a great topic. There hasn't been any discussion in this project 
to date on this, mostly because everyone has been busy on the bring up 
of user mode threads and all the issues around that. So for now at 
least, a parallel stream will submit to the FJ common pool. That will be 
right for existing code operating on data, it might not be right if 
there are stream operations doing networking I/O. Whether the choice to 
use fibers is implicit or explicit isn't clear. There is a potential 
mini project here, esp. with Brian's comment that there may be different 
splitting choices.

Someone else brought up CompleteableFuture where the xxxAsync will use 
the FJ common pool when an Executor isn't specified specified. A 
suggestion at one point was to add variants that would schedule a fiber 
and that would at least be explicit in the API. This is another topic 
that will need attention at some point.

-Alan.