hg: lambda/lambda/jdk: Replacement for VariableStreamBuilder that doesn't do re-allocs. Better chunk sizing will come in future updates.

[Peter Levart]

On 10/05/2012 03:26 AM, mike.duigou at oracle.com wrote:
> Changeset: f673f3d4682f
> Author:    mduigou
> Date:      2012-10-04 18:26 -0700
> URL:       http://hg.openjdk.java.net/lambda/lambda/jdk/rev/f673f3d4682f
>
> Replacement for VariableStreamBuilder that doesn't do re-allocs. Better chunk sizing will come in future updates.
>
> ! src/share/classes/java/util/streams/StreamBuilders.java
> ! test-ng/tests/org/openjdk/tests/java/util/streams/ops/StreamOpTestCase.java
>
>

Hi Mike and others,

Interestingly, I have been playing with the idea about an alternative 
StreamBuilder implementation in recent days. It's basically the same as 
your's SimpleSpinedStreamBuilder with an additional feature: an int[] 
chunkSizes array maintained in parallel with T[][] chunks, such 
StreamBuilder can optimize the .addAll(Collection) method when it 
encounters it's own pair as an argument by accepting the already built 
chunks from the argument SB into it's own arrays. This avoids 
re-building the chunks by appending individual elements by allowing 
intermediary chunks in the SB not be fully populated.

This is the simplified idea that can be further refined by for example 
maintaining the invariant that any two consecutive chunks that can be 
merged into one chunk are merged as part of any modification operation.

Such SB can optimize the reduction phase of the GroupByOp parallel 
operation when groups become large or any reduction phase that just 
collects the results from individual mapping tasks.

I already have some code with limited functionality (just appending to 
Collection is supported, no deletes) and can contribute if interested...

Regards, Peter