Draft JEP for Http client in JDK 9

[Michael McMahon]

Hi,

You might remember a new Http client API was proposed for JDK 8, but the 
work
was dropped from that project. So, we would like to return to the idea 
for JDK 9.
The following is the text of a draft JEP for the work. Comments welcome.

Thanks,
Michael


Title: New HTTP Client
Author: Michael McMahon

Summary
-------
Define and implement a new Http client API to eventually replace the legacy
HttpURLConnection and which also implements HTTP 2.0 and websockets.

Motivation
----------
Problems with the existing API and implementation:

+ URLConnection based API was designed with multiple protocols in mind,
   nearly all of which are defunct now (ftp, gopher, etc.)

+ predates HTTP 1.1 and is too abstract

+ hard to use (much behavior undocumented)

+ works in blocking mode only (one thread per request/response)

+ very hard to maintain

Goals
-----
+ API must be easy to use for common cases including a simple blocking mode.

+ also need an asynchronous mode which decouples requests and responses
   and which allows for notification of events like "headers received", 
errors,
   "response body received"

+ modernize HTTP support (up to Http 2.0)

+ allow for alternative implementations

+ simple and concise API which caters for 80-90% of application needs

+ must expose all relevant aspects of the HTTP protocol request to a server
   and response from a server (headers, body, status codes etc)

+ must include standard and common authentication mechanisms

+ must be able to easily set up the web sockets handshake

+ must support HTTP 2.0 (application level semantics of 2.0 are mostly the
   same as 1.1, though wire protocol completely different)
    + ability to negotiate upgrade from 1.1 to 2.0 (or not), or select 2.0
      from start
    + server push (ability of server to push resources to client without 
being
      requested by client)

+ must perform security checks consistent with existing networking classes

+ should be friendly towards new language features like lambdas


Non goals
---------
This work is targeted for JDK 9 only. It is not intended to be back 
portable
to previous releases. While this API is intended to replace the 
URLConnection
API for new code, it is not intended to immediately re-implement the old 
API
using the new API/implementation. This may happen as future work.

These are also some requirements which had been considered in JDK 8, but
in the interests of keeping the API as simple as possible, are being 
left out
for JDK 9. Some of these will assume less importance with the gradual 
adoption
of http 2.0 eg. connection caching because there is less requirement for
multiple TCP connections with Http 2.0.

+ request/response filtering
+ API for pluggable connection cache
+ general upgrade mechanism

Description
-----------
Some prototyping work was done during JDK 8 and separate classes were 
defined
for the http client, requests and responses. The builder pattern was used
to separate mutable entities from the immutable products. The prototype was
built on NIO AsynchronousSocketChannels and that API will be the 
starting point
for this work. It is expected that the API will undergo a number of 
iterations
before being finalised. The API/implementation was also standalone,
ie. existing stack will be left as is to ensure compatibility
and allow a phased approach (don't have to support everything at once).

The prototype API also included support for:

+ separate request/response (like servlet and http server API)

+ Asynchronous notification handlers will be defined for following events:
     + response headers received
     + response error
     + response body received
     + server push (2.0 only) means "here is a request for which the server
       will send a response"

+ https via SSLEngine
+ proxying
+ cookies
+ authentication


Alternatives
------------

A number of existing Http client API and implementations exist: for example
Jetty (org.mortbay.jetty) and the Apache Http client.

Both of these are both rather heavy-weight in terms of numbers of packages
and classes and don't take advantage of newer language features like 
lambdas.

Risks and Assumptions
---------------------
Https 2.0 support depends on TLS (Application Layer Negotiation Extension)
which is not currently supported in JDK.

The Http 2.0 spec itself is still in internet-draft form, but is 
expected to
be submitted as a draft standard Nov 2014.

API surface could grow if additional features added.

Impact
------

+ Other JDK components: no impact
+ Compatibility: no impact on existing API, but new tests required for 
new API
+ Security: will need careful audit
+ Portability: will be no native code in implementation
+ User Interface: no impact
+ Documentation: no impact
+ Internationalization: no impact
+ Localization: no impact
+ Legal: no impact
+ Other: no impact

Testing
-------

Will need new tests. The internal http server will provide a suitable test
harness for regression tests and TCK tests. SQE tests could use it also,
but may need to test against real http servers.