[java.net.http.HttpClient] Active monitoring of resolved IP addresses

Wed Jul 29 12:20:55 UTC 2020

Hi there,

I haven't got news about this subject but I have built a library that works
around the problem Github resilient-httpclient
<https://github.com/nhenneaux/resilient-httpclient>.
It overrides the HTTP host header plus the SNI field in TLS handshake. It
also provides custom TLS server name verification. I made a small
presentation
<https://docs.google.com/presentation/d/1ixrKR79pX5jDGRO46mA03r20n3sQGhu2TRYoe_uKFYI/edit#slide=id.g76517f9773_0_278>
with more details about it.

The integration is a bit hacky and might probably be improved by providing
an explicit API to customize the host header and TLS server name. Would you
have any idea how it could be integrated inside OpenJDK? I would be able to
help if you think it is valuable.

I also had to disable some flags in the JVM to have it fully functional.
Perhaps it would be better to have an API to build an HttpClient with those
checks disabled at instance level instead of globally?

System.setProperty("jdk.internal.httpclient.disableHostnameVerification",
Boolean.TRUE.toString());
System.setProperty("jdk.httpclient.allowRestrictedHeaders", "host");

Thanks in advance for your feedback,

Best regards,

Nicolas Henneaux
nicolas.henneaux at gmail.com

On Tue, 17 Dec 2019 at 13:32, Nicolas Henneaux <nicolas.henneaux at gmail.com>
wrote:

> Hi Daniel,
>
> Have you got the chance to look at my previous mail?
>
> Thank you in advance,
>
> Best regards,
>
> Nicolas
>
> On Sun, 17 Nov 2019 08:49 Nicolas Henneaux, <nicolas.henneaux at gmail.com>
> wrote:
>
>> Hi Daniel,
>>
>> Thank you for your answer !
>>
>> Such mechanism to provide an *InetSocketAddress* from a hostname would
>> be a great improvement and matches the usage I have.
>>
>> I have an implementation using the IP in the URL in
>> https://github.com/nhenneaux/resilient-httpclient/tree/master/single-host-httpclient.
>> It needs a custom TLS hostname verifier which is not that nice but I could
>> live with it.
>> The main problem I have with such strategy is the loose of the hostname
>> in the HTTP header which causes wrong routing on the server side. Indeed
>> the target server relies on the host header for routing to the right
>> application. I had actually to target a server using such mechanism that
>> doesn't work with IP in the URl unfortunately.
>> I think it could also cause problem for SNI since IP's wouldn't be able
>> to match the right certificate.
>>
>> Another solution could be to provide a mechanism to filter out a list of
>> *InetSocketAddress.* From *N* address resolved for the target server
>> (proxy or actual server), it returns one address that will be used to
>> create the socket and send the payload (with proxy protocol or regular
>> HTTP).
>>
>> If you decide to provide an extension mechanism what would be the next
>> steps? Tell me if I can help in any way to move forward on this subject.
>>
>> Thanks for your help anyway,
>>
>> Best regards,
>>
>> Nicolas Henneaux
>>
>>
>> On Fri, 15 Nov 2019 at 17:18, Daniel Fuchs <daniel.fuchs at oracle.com>
>> wrote:
>>
>>> Hi Nicolas,
>>>
>>> Sorry for the late reply...
>>>
>>> On 07/11/2019 17:13, Pavel Rappo wrote:
>>> >> On 7 Nov 2019, at 14:24, Nicolas Henneaux <nicolas.henneaux at gmail.com>
>>> wrote:
>>> >>
>>> >> Hi,
>>> >>
>>> >> *I am trying to build an HTTP client based on
>>> java.net.http.HttpClient and
>>> >> I would like to have some control on the IP addresses resolved under
>>> the
>>> >> hood.*
>>> >>
>>> >> I would like such mechanism to use all the IP addresses behind the
>>> host of
>>> >> the request. The underlying problem, I am trying to solve, is to
>>> react as
>>> >> soon as possible to an unavailability of a target IP. It could be
>>> done by
>>> >> doing active monitoring for each IP and use only the addresses
>>> available
>>> >> (at TCP, TLS or HTTP level) when sending an HTTP request. I also
>>> would like
>>> >> to load balance the traffic between all the IP’s and actively detect
>>> change
>>> >> in the DNS resolution. I am maintaining a long live connection to an
>>> >> external HTTPS service.
>>> >>
>>> >> At the moment, only one IP address is used by the HTTP java client
>>> and pool
>>> >> of connections is maintained to target this IP.
>>> >>
>>> >> I have built a work-around by using the IP address in the URL
>>> provided to
>>> >> the HTTP client and using a specific trust manager matching the
>>> expected
>>> >> name to keep the TLS validation. It has several limitation. In
>>> particular
>>> >> it changes the HTTP request which could impact the response you
>>> received
>>> >> depending on the server you target. It also forces to use one HTTP
>>> client
>>> >> by host you want to target. I have created a small GitHub repository
>>> >> <https://github.com/nhenneaux/resilient-httpclient> to illustrate
>>> this
>>> >> workaround.
>>> >>
>>> >> I explored a bit the HTTP client implementation and so far I have not
>>> found
>>> >> any way to implement such mechanism. In particular, I am stuck in
>>> >> jdk.internal.net.http.HttpRequestImpl#getAddress which is package
>>> visible
>>> >> and prevents any extension.
>>> >>
>>> >> Such mechanism exists in Jetty HTTP client (see
>>> >> org.eclipse.jetty.util.SocketAddressResolver) or in HTTP Apache
>>> client.
>>> >>
>>> >> *Is there any existing mechanism I could use to properly solve such
>>> >> problem? If none, is there a way to contribute or to submit a request
>>> of
>>> >> improvement? *
>>>
>>> I don't think there is any existing mechanism, besides hardcoding
>>> the server's address in the URL. One thing we could potentially
>>> consider for future evolution is add an API to pass a lambda to the
>>> HttpClient.Builder that would take a hostname and a port and return
>>> an InetSocketAddress.
>>> The HttpRequestImpl could use that when its getAddress() method
>>> is called. Note however that this would only help when no proxy is
>>> used (except maybe in case of tunneling?).
>>>
>>> Would that be enough to allow you to implement your logic?
>>>
>>> There are security implications (permission checking) that would
>>> need to be carefully considered however, if we decided to that.
>>>
>>> On the other hand having the server address in the URL instead
>>> should work even in the presence of proxies.
>>>
>>> best regards,
>>>
>>> -- daniel
>>>
>>> >>
>>> >>
>>> >>
>>> >> Thank you in advance for your help!
>>> >>
>>> >>
>>> >>
>>> >> Best regards,
>>> >>
>>> >>
>>> >>
>>> >> *Nicolas Henneaux*
>>> >
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/net-dev/attachments/20200729/ef9ec16b/attachment.htm>