StreamBuilder is awesome

[Paul Sandoz]

On Apr 12, 2013, at 11:09 AM, Ali Lahijani <alahijani at gmail.com> wrote:

> The underlying implementation can do a good splitting job only if individual items in the stream take the same amount of time to process. If they are of variable size, that should also be taken account of in determining split points.
> 
> Say you are processing a list of binary files, given as a File[]. A File takes more time to filter(), map(), etc. if it's larger. So, Arrays.stream() gives you a "balanced" Stream<File> only if the files are of the same relative size. If they are not, you can do something like this:
> 
> int chunkSize = 0;
> for (Fie file: files) {
>     builder.accept(file);
> 
>     chunkSize += file.length();
>     if (chunkSize > threshold) {
>         builder.split();
>         chunkSize = 0;
>     }
> }
> 
> This is much easier than the only other option, writing a custom Spiterator.
> 

Having the developer provide hints at this level on how to split is asking for trouble IMHO. You cannot really control the splitting; all the above does is hint at is degenerate splitting (right-balanced tree).

If you want to control how things are split you need to write a Spliterator :-) so you can control the balance *and* dictate when splitting should not occur.

Paul.