Concerns about parallel streams
Howard Lovatt
howard.lovatt at gmail.com
Thu Jul 11 17:10:12 PDT 2013
parrallel(boolean) could be provided since at present if you have some
variability in either N or Q you have to do:
Stream s = ....
if (...) { s.parallel(); }
s....
which is a pain.
On Friday, July 12, 2013, Doug Lea wrote:
> On 07/11/13 17:08, Sam Pullara wrote:
>
>> Hoping Doug enters the thread soon….
>>
>
> (It's great to feel needed, but today maybe a little too much :-)
>
> A couple of quick notes:
>
> If you are writing from-scratch ForkJoin programs
> rather than stream-based ones, you become immediately
> aware that you have to make some decisions about task
> granularity and partitioning. The rule of thumb we state in
> FJ is if you stay above a thousand or so instructions per
> leaf task, you'll have a good chance of success.
>
> The big problem when you automate this via streams is that
> most programmers have nearly no idea about any of the
> components of this otherwise straightforward guidance.
> And as Aleksey explained, there are few prospects for
> magically automating in general.
> Yet any attempt at providing any form of parameterization
> of hinting to control this has been defensibly rejected.
> As a result we have a completely opaque cost model.
> No sense in pretending otherwise.
>
> Despite this, the easy guidance is:
>
> If you have a lot of data, or very costly per-element computations,
> the best practice is to use parallel(). Otherwise, feel free to
> experiment with it, but don't expect any miracles.
>
> We could even give factor-of-1000-proof numbers here:
> A million elements. A million instructions
>
> -Doug
>
>
>
--
-- Howard.
More information about the lambda-dev
mailing list