Combining streams

[Paul Sandoz]

Hi Michael,

These are good questions, what we are currently lacking is a description of the Stream API to help build up a mental model, however i am not sure JavaDoc is the best place for that.


On Jul 26, 2013, at 3:52 AM, Michael Hixson <michael.hixson at gmail.com> wrote:

> I have a couple of questions about combining streams.  Setting aside
> the question of which code is easier to read, I am wondering about
> things like performance or other hidden gotchas.
> 
> 1. To combine two streams, I should do this:
> 
> Stream.concat(s1, s2)
> 
> not this:
> 
> Stream.of(s1, s2).flatMap(x -> x)
> 
> ... right?
> 

The former will preserve some properties of both streams if it can, specifically if both streams are sized the output is sized and certain optimizations can kick in, where as flatMap will clear any size properties and thus such optimizations are no longer applicable.

The flatMap operation is also less efficient since it requires more work per element.


> 2. To combine more than two streams, I should do this:
> 
> Stream.of(s1, s2, s3, ...).flatMap(x -> x)
> 
> not this:
> 
> Stream.of(s1, s2, s3, ...).reduce(Stream.empty(), Stream::concat)
> 
> ... right?
> 

This one might be somewhat trickier to evaluate in terms of performance. I find the former more elegant, but as Brian points out it will not work for infinite streams, since the implementation does s.forEach(...)

Sequentially the latter is equivalent to:

   Stream.concat(Stream.concat(Stream.concat(Stream.empty(), s1), s2), s3)

So there can be lots of wrapping going on. For parallel evaluation the wrapping is likely to be more prevalent, since it will mirror that of the computation tree. 

IIRC the EG may have touched upon a var-arg concat, but i suspect it fell through the gaps.

Paul.