GroupByOp.evaluateParallel

[Brian Goetz]

Another implementation approach would be to only have one Map, a ConcurrentHashMap, whose keys are StreamBuilders.  The first thread to discover a key does a putIfAbsent(k, new SB()).  Insertion proceeds as 

  synchronized(sb) {
    sb.add(v);
  }

If the classifier function spreads the keys broadly then contention may not be a huge issue, and then there is no reduction phase at all. 


On Oct 3, 2012, at 11:27 AM, Paul Sandoz wrote:

> Hi,
> 
> Here is an implementation of GroupByOp.evaluateParallel:
> 
>  http://cr.openjdk.java.net/~psandoz/lambda/pargroupby/webrev/
> 
> based from the previous patches in my queue.
> 
> The casts from Map<K, StreamBuilder<T>> to Map<K, Collection<T>> are annoying. Perhaps we need a more formal transition from StreamBuilder to Collection?
> 
> The reduction phase (traversing up the computation tree) could be more efficient if two or more StreamBuilder<T> instances could be efficiently merged, perhaps if we know something about the implementation data structures. This feels very like what TreeUtils.Node does.
> 
> Potentially there could be another parallel evaluation if we wanted to "flatten" the tree of values to T[].
> 
> Paul.
>