A parallel HttpClient sendAync question
Weijun Wang
weijun.wang at oracle.com
Tue Nov 13 02:35:07 UTC 2018
I'm scanning a file and downloading links inside:
lines.flapMap(x -> Stream.ofNullable(findURIFrom(x)))
.map(l -> download(c, l))
.forEach(f -> f.join());
CompletableFuture<HttpResponse<Path>> download(HttpClient c, URI link) {
return c.sendAsync(HttpRequest.newBuilder(link).build(),
HttpResponse.BodyHandlers.ofFile(Path.of(link.getPath())));
}
However, it seems the download is one by one and not parallel. I guess maybe map() is lazy and each request is only send when forEach() is called and the next one is only sent after join().
I can only collect the jobs into a list and then call join() on CompletableFuture.allOf(list). Is there a simpler way?
Thanks
Max
More information about the net-dev
mailing list