A parallel HttpClient sendAync question

Weijun Wang weijun.wang at oracle.com
Tue Nov 13 02:35:07 UTC 2018


I'm scanning a file and downloading links inside:

lines.flapMap(x -> Stream.ofNullable(findURIFrom(x)))
     .map(l -> download(c, l))
     .forEach(f -> f.join());

CompletableFuture<HttpResponse<Path>> download(HttpClient c, URI link) {
    return c.sendAsync(HttpRequest.newBuilder(link).build(),
            HttpResponse.BodyHandlers.ofFile(Path.of(link.getPath())));
}

However, it seems the download is one by one and not parallel. I guess maybe map() is lazy and each request is only send when forEach() is called and the next one is only sent after join().

I can only collect the jobs into a list and then call join() on CompletableFuture.allOf(list). Is there a simpler way?

Thanks
Max


More information about the net-dev mailing list