RFR [11] 8197564: HTTP Client implementation - JEP 321

[James Roper]

Hi Simone,

There are some RS implementations that offer a user facing API that only
offers their equivalent of a Publisher to end users, but that is immaterial
to RS, since RS is not an end user facing API, but an integration API. End
users should not be implementing their own Publisher/Subscriber instances,
so it doesn't matter about how difficult they are or aren't to implement.
RS compliant implementations should have no problems offering either
subscribers or publishers, this is why the RS TCK offers verification for
both.

I have been involved in the implementation of several RS implementations,
including:

* Akka Streams
* netty-reactive-streams:
https://github.com/playframework/netty-reactive-streams/
* A servlet implementation of reactive streams:
https://github.com/jroper/reactive-streams-servlet
* An iteratees based implemnetation of reactive streams:
https://github.com/playframework/play-iteratees

Plus, at Lightbend we're also working on an API that is intended to be a
proposal for the JDK that offers basic Reactive Streams transformation APIs:

https://github.com/lightbend/reactive-streams-utils/

*None* of the above APIs are publisher biased, in fact some of them are
even subscriber biased, and they present no problems when implementing the
RS APIs. Furthermore, the JDK library proposal also has an implementation
using RxJava2, which demonstrates that RxJava, when using Reactive Streams
as an integration API, is perfectly capable of exposing and delivering
Subscribers as per the spec.

I think the problem here is that some libraries have confused the end user
API with the integration API. They are not meant to be the same thing. When
Lightbend approached the Netflix devs working on RxJava to start the
Reactive Streams effort, it was never the intention that we would create a
common end user API, it was the intention that we would create an API that
would allow our respective streaming libraries to integrate with each
other. Reactive Streams is the result. That RxJava has confused these two
things is disappointing, but I think it very odd that we would take the
constraints of the RxJava and Reactor end user APIs, and force
implementations to conform to how they would like all users to use it, even
though most other RS implementations don't do that. That makes no sense for
an integration API.

Regards,

James

On 31 March 2018 at 00:34, Simone Bordet <simone.bordet at gmail.com> wrote:

> Hi,
>
> On Wed, Mar 21, 2018 at 9:33 PM, Chris Hegarty <chris.hegarty at oracle.com>
> wrote:
> > Hi,
> >
> > This is a request for review for the HTTP Client implementation - JEP
> 321.
>
> It is my understanding, and that of the ReactiveStreams (RS) people I
> spoke with, that both the RS API and the higher level API based on RS
> (such as those offered by Reactor and RxJava2) are supposed to expose
> to users only Publishers.
> This may be counterintuitive at beginning: the typical case of
> InputStream to read content and OutputStream to write content are both
> represented with Publishers in the RS experience.
> The reason behind only offering Publishers is that Subscribers are
> more difficult to implement by regular developers; Processors even
> more so, so it is better that this task is left to library
> implementers such as the JDK, Reactor and RxJava2.
>
> It is therefore a surprise that the the HTTP client uses instead
> Subscriber for the response content.
> BodyHandler must return a BodySubscriber, which is-a Flow.Subscriber.
>
> This will force users that need to write non-trivial response content
> handling to implement a Subscriber, or most of the times a Processor
> (to convert the Subscriber back into a Publisher for consumption by
> other more "regular" RS APIs).
> It also creates an asymmetry between read and write side, so that for
> example it is not possible, without writing a processor, to echo a
> response content as the next request content.
> Another difficult example is piping the response content to a
> WebSocket RS API (that would take a Publisher), and so on for
> basically all RS libraries out there.
>
> I would like to suggest to review this: it is definitely possible to
> have BodyHandler refactored in this way:
>
> public Publisher<T> apply(int statusCode, HttpHeaders responseHeaders,
> Publisher<ByteBuffer> responseContent);
>
> Returning a Publisher that performs transformations on the
> responseContent Publisher is the heart of libraries such as Reactor
> and RxJava2, so this will be super simple for users of the API.
> Utility Publishers that currently discard, convert to string, save to
> file, etc. can be equally trivially implemented as they are now in the
> JDK.
>
> With the occasion, I think that the statusCode parameter and the
> responseHeaders parameter can be replaced by a single HttpResponse
> parameter, which would be better for future maintenance in case other
> information needs to be provided.
> The simple case that comes to mind is the HTTP version of the
> response, which may be different from that of the request and
> indicates server capabilities: this is currently not provided in the
> BodyHandler.apply() signature.
>
> So perhaps:
>
> public Publisher<T> apply(HttpResponse response, Publisher<ByteBuffer>
> responseContent);
>
> This change would also simplify the maintenance since BodySubscriber
> will be removed.
>
> My concern is that driving users of the JDK APIs outside of the
> familiar RS APIs offered by Reactor and RxJava2 where only Publishers
> are relevant, by forcing them to implement Subscribers and Processors
> (heck, I even have an IntelliJ warning telling me that I should not do
> that!), will cause confusion and be potential source of bugs - we all
> know non-blocking/async programming is hard.
>
> Thanks !
>
> --
> Simone Bordet
> ---
> Finally, no matter how good the architecture and design are,
> to deliver bug-free software with optimal performance and reliability,
> the implementation technique must be flawless.   Victoria Livschitz
>



-- 
*James Roper*
*Senior Octonaut*

Lightbend <https://www.lightbend.com/> – Build reactive apps!
Twitter: @jroper <https://twitter.com/jroper>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/net-dev/attachments/20180402/38e17fd5/attachment.html>