[External] : Re: My expirience with gatherers

[Viktor Klang]

Hi Olexandr,

It's not a correction 🙂 Adding greedy won't make any semantical difference (if you are not trying to short-circuit) but it will allow some optimizations since the evaluation of the Gatherer doesn't need to check for short-circuiting on a per-element basis.

Cheers,
√


Viktor Klang
Software Architect, Java Platform Group
Oracle

Confidential – Oracle Internal
________________________________
From: Olexandr Rotan <rotanolexandr842 at gmail.com>
Sent: Monday, 20 October 2025 10:32
To: Viktor Klang <viktor.klang at oracle.com>
Cc: core-libs-dev <core-libs-dev at openjdk.org>
Subject: [External] : Re: My expirience with gatherers

Hi Viktor!

Thanks for your insights, those are really valuable. I went through the usages around the codebase and actually found out that any pipeline using this gatherers actually ended up with either toList or collect(Collectors.toUnmodifiableSet()), that is probably why I have never noticed something off around short circuiting, though implementations are clearly are not designed right in this terms. Thank you for the correction!

Best regards

On Mon, Oct 20, 2025 at 11:22 AM Viktor Klang <viktor.klang at oracle.com<mailto:viktor.klang at oracle.com>> wrote:
Thank you for sharing your practical experience with Gatherers, Olexandr—it is much appreciated!

Some thoguhts while reading your email:


  *
Given that your custom Gatherers shown never short-circuit, you might find it performance-wise valuable to instantiate then with Gatherer.Integrator.ofGreedy.


  *
For your finishers, I've personally found it valuable to short-circuit of the downstream push returns false, so in the cases above it'd be something like

        If (!downstream.push(element))
            return;

Cheers,
√


Viktor Klang
Software Architect, Java Platform Group
Oracle

Confidential – Oracle Internal
________________________________
From: core-libs-dev <core-libs-dev-retn at openjdk.org<mailto:core-libs-dev-retn at openjdk.org>> on behalf of Olexandr Rotan <rotanolexandr842 at gmail.com<mailto:rotanolexandr842 at gmail.com>>
Sent: Friday, 17 October 2025 20:45
To: core-libs-dev <core-libs-dev at openjdk.org<mailto:core-libs-dev at openjdk.org>>
Subject: My expirience with gatherers

Greetings to the core libs folks. I have been using Gatherers extensively for my job in a past few months, and would like to share some of the gatherers that I have been extensively using, so maybe some of them may be a source of inspiration for evolving the Gatherers class.

1. eagerlyConsume()
Implementation:

public static <T> Gatherer<T, ?, T> eagerlyConsume() {
    return Gatherer.of(
        ArrayList<T>::new,
        (list, val, downstream) -> {
            list.add(val);
            return true;
        },
        (left, right) -> {
            left.addAll(right);
            return left;
        },
        (list, downstream) -> {
            for (var item : list) {
                downstream.push(item);
            }
        }
    );
}

Purpose: many times, i need to perform a concurrent mapping of jpa entities to dtos. Unfortunately, mapConcurrent does not accept custom executors, which i need in order to propagate auth, transaction and other contexts. So, therefore, I previously have used following pattern:

stream().map(COmpletableFuture.supplyAsync(..., executor)).toList().stream().map(CompletableFuture::join)

toList is required here to eagerly start all futures, as otherwise the will actually launch sequentially due to the pulling nature of streams. With gatherer, on the other hand, i can achieve following:
stream().map(COmpletableFuture.supplyAsync(..., executor))..gather(eagerlyConsume()) .map(CompletableFuture::join), which looks much more readable, and (presumably, haven't actually verified it) should have better performance

2. ofCollector
Implementation:

public <T, A, R> Gatherer<T, A, R> ofCollector(Collector<T, A, R> collector) {
    return Gatherer.of(
        collector.supplier(),
        (a, t, _) -> {
            collector.accumulator().accept(a, t);
            return true;
        },
        collector.combiner(),
        (state, downstream) -> downstream.push(collector.finisher().apply(state))
    );
}

Pretty self explanatory, this is just an adapter of collector to gatherer, allowing arbitrary collector-defined folds

3. collectThenFlatten & co
Implementations:

public static <T, A, R extends Collection<T>> Gatherer<T, A, T> collectThenFlatten(Collector<T, A, R> collector) {
    return Gatherer.of(
        collector.supplier(),
        (a, t, _) -> {
            collector.accumulator().accept(a, t);
            return true;
        },
        collector.combiner(),
        (state, downstream) -> {
            for (var item : collector.finisher().apply(state)) {
                downstream.push(item);
            }
        }
    );
}

public static <T, A, K, V, R extends Map<K, V>> Gatherer<T, A, Map.Entry<K, V>> collectThenFlattenEntries(Collector<T, A, R> collector) {
    return Gatherer.of(
        collector.supplier(),
        (a, t, _) -> {
            collector.accumulator().accept(a, t);
            return true;
        },
        collector.combiner(),
        (state, downstream) -> {
            for (var entry : collector.finisher().apply(state).entrySet()) {
                downstream.push(entry);
            }
        }
    );
}

These are more specialized adapters of collector adapters, mostly a convenience for avoiding flatMapping results, THough, I would like to note that collectThenFlattenEntries is mostly used specifically with groupingBy collector, to avoid following nasty chains:

collect(groupingBy(...)).entrySet().stream()

Maybe it's just my personal preferences, but i really dislike this back n forth from stream to map, then to set and to stream again, so this gatherer seems pretty pleasant to use

That's basically all that I wanted to share regarding this topic, hope this experience will have some value for core libs maintainers

Best regards
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/core-libs-dev/attachments/20251020/0a08a7e5/attachment-0001.htm>