RFR: 8319123: Implement JEP 461: Stream Gatherers (Preview) [v13]

[Svein Otto Solem]

Tested the Gatherer implementation, and it is a very nice work.

I have always wanted a way of writing my own stream operations and also have incomplete composable pipelines which could be combined in different ways. Very interesting.

The four Gatherer functions - initializer/integrator/combiner/finisher - all return function objects.
In the javadoc for these Gatherer functions, no requirements are described for these functions.
(though API note on class level says "Each invocation of initializer(), integrator(), combiner(), and finisher() must return a semantically identical result.")

It would be nice to have the requirements stated on the function level also. Looking at the class level javadoc is not always done.

How often and in which sequence these functions are called are of interest, especially for stateful Gatherers.
I created a simple gatherer which prints some trace info for these functions (see below).
It is tested using branch "pr/16420" of the jdk pr. 21.nov.
   - The "initializer()" function are in some cases called twice from the library.
   - The "initializer().get()" function is called only once in the tested scenarios.
     This is expected since the method is called "initializer", but this behaviour could have been documented.
   - The sequence these functions are called varies from use case to use case, sometimes the "integrator()" is called before "initializer()".
     Sometimes the "finisher()" are called before first call to "integrator()", sometimes after.


I've marked the unexpected/unneccesary/repeated calls with ? marks.

All this may be correct, but is it ?


------------------- Trace --------------------

Using andThen                   Using separate gathers                Using separate gathers with map(identity)
Creates WithIndex(A)            Creates WithIndex(A)                  Creates WithIndex(A)
Creates WithIndex(B)            Creates Integrator(A)                 Creates Integrator(A)
Creates WithIndex(C)            Creates WithIndex(B)                  Creates WithIndex(B)
Creates Initializer(A)          Creates Initializer(A)                Creates Integrator(B)
Creates Integrator(A)          ?Creates Integrator(A)                 Creates WithIndex(C)
Creates Finisher(A)             Creates Finisher(A)                   Creates Integrator(C)
Creates Initializer(B)          Creates Initializer(B)                Calls Collectors.toList
Creates Integrator(B)           Creates Integrator(B)                 Finished Collectors.toList
Creates Finisher(B)             Creates Finisher(B)                  ?Creates Integrator(C)
Creates Initializer(C)          Creates WithIndex(C)                 ?Creates Integrator(B)
Creates Integrator(C)          ?Creates Initializer(A)               ?Creates Integrator(A)
Creates Finisher(C)            ?Creates Integrator(A)                 Creates Initializer(A)
Calls Collectors.toList        ?Creates Finisher(A)                   Invokes Initializer(A)
Finished Collectors.toList     ?Creates Initializer(B)                Creates Initializer(B)
Invokes Initializer(A)         ?Creates Integrator(B)                 Invokes Initializer(B)
Invokes Initializer(B)         ?Creates Finisher(B)                   Creates Initializer(C)
Invokes Initializer(C)          Creates Initializer(C)                Invokes Initializer(C)
Invokes Integrator(A)           Creates Integrator(C)                 Invokes Integrator(A)
Invokes Integrator(B)           Creates Finisher(C)                   Invokes Integrator(B)
Invokes Integrator(C)           Calls Collectors.toList               Invokes Integrator(C)
Invokes Integrator(A)           Finished Collectors.toList            Invokes Integrator(A)
Invokes Integrator(B)           Invokes Initializer(A)                Invokes Integrator(B)
Invokes Integrator(C)           Invokes Initializer(B)                Invokes Integrator(C)
Invokes Integrator(A)           Invokes Initializer(C)                Invokes Integrator(A)
Invokes Integrator(B)           Invokes Integrator(A)                 Invokes Integrator(B)
Invokes Integrator(C)           Invokes Integrator(B)                 Invokes Integrator(C)
Invokes Finisher(A)             Invokes Integrator(C)                 Creates Finisher(A)
Invokes Finisher(B)             Invokes Integrator(A)                 Invokes Finisher(A)
Invokes Finisher(C)             Invokes Integrator(B)                 Creates Finisher(B)
                                Invokes Integrator(C)                 Invokes Finisher(B)
                                Invokes Integrator(A)                 Creates Finisher(C)
                                Invokes Integrator(B)                 Invokes Finisher(C)
                                Invokes Integrator(C)
                                Invokes Finisher(A)
                                Invokes Finisher(B)
                                Invokes Finisher(C)

----- Java code --------------------

package org.example;

import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.BiConsumer;
import java.util.function.Supplier;
import java.util.stream.Collector;
import java.util.stream.Collectors;
import java.util.stream.Gatherer;
import java.util.stream.Stream;

import static java.lang.StringTemplate.STR;
import static java.util.function.Function.identity;

public class GatherEx {

    static <T> Gatherer<T, AtomicInteger, Indexed<T>> withIndex(String name) {
        return new WithIndexGatherer<>(name);
    }

    static class WithIndexGatherer<T> implements Gatherer<T, AtomicInteger, Indexed<T>> {
        final String name;

        public WithIndexGatherer(String name) {
            System.out.println(STR. "Creates WithIndex(\{ name })" );
            this.name = name;
        }

        @Override
        public Supplier<AtomicInteger> initializer() {
            System.out.println(STR. "Creates Initializer(\{ name })" );
            return () -> {
                System.out.println(STR. "Invokes Initializer(\{ name })" );
                return new AtomicInteger(0);
            };
        }

        @Override
        public Integrator<AtomicInteger, T, Indexed<T>> integrator() {
            System.out.println(STR. "Creates Integrator(\{ name })" );
            return (state, element, downstream) ->
            {
                System.out.println(STR. "Invokes Integrator(\{ name })" );
                return downstream.push(new Indexed<>(state.getAndIncrement(), element));
            };
        }

        @Override
        public BiConsumer<AtomicInteger, Downstream<? super Indexed<T>>> finisher() {
            System.out.println(STR. "Creates Finisher(\{ name })" );
            return (_, _) -> {
                System.out.println(STR. "Invokes Finisher(\{ name })" );
                Gatherer.super.finisher();
            };
        }
    }

    record Indexed<T>(int i, T data) {
    }

    public static void main(String[] args) {
        System.out.println("Using andThen");
        System.out.println(Stream.of(1, 2, 3)
                .gather(withIndex("A")
                        .andThen(withIndex("B"))
                        .andThen(withIndex("C")))
                .collect(createCollector()));
        System.out.println("Using separate gathers");
        System.out.println(Stream.of(1, 2, 3)
                .gather(withIndex("A"))
                .gather(withIndex("B"))
                .gather(withIndex("C"))
                .collect(createCollector()));
        System.out.println("Using separate gathers with map(identity)");
        System.out.println(Stream.of(1, 2, 3)
                .map(identity())
                .gather(withIndex("A"))
                .map(identity())
                .gather(withIndex("B"))
                .map(identity())
                .gather(withIndex("C"))
                .map(identity())
                .collect(createCollector()));

    }

    private static Collector<? super Indexed<?>, ?, List<Indexed<?>>> createCollector() {
        System.out.println("Calls Collectors.toList");
        try {
            return Collectors.toList();
        } finally {
            System.out.println("Finished Collectors.toList");
        }
    }
}


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/core-libs-dev/attachments/20231124/2d46eb22/attachment-0001.htm>