6370908: Add support for HTTP_CONNECT proxy in Socket class

[Christopher Hegarty -Sun Microsystems Ireland]

Hi Damjan,

After spending more time looking at this, I still haven't found an 
elegant solution for supporting authentication.

You've correctly identified an issue with the fact we have no guaranteed 
of a persistent connection with the proxy. Changing the socket identity 
is not as simple as overriding getIn/OutputStream since other methods 
like setting/getting options, close, shutdown, etc use the impls fd 
field. This field needs to refer to the correct file descriptor object.

There is quite a lot of logic in the http protocol handler to support 
authentication and looking at it again it is closely tied to the handler 
implementation itself. I would prefer to reuse this code if possible.

Have you made any progress on this? If we want authentication it may be 
more straight forward to revert back to your original proposal of using 
HttpURLConnection directly and having its socket exposed. Though, I'm 
still not convinced of this solution either.

-Chris

Damjan Jovanovic wrote:
> Hi Chris
> 
> I think that without authentication, you'll just get another RFE
> asking for authentication. Many corporate proxies require
> authentication, especially for a liberal request like CONNECT.
> 
> java.net.Socket seems to delegate its impl field's methods. If we have
> another, internal socket in HttpConnectSocketImpl, and we override
> getInputStream() and getOutputStream() in HttpConnectSocketImpl to
> return this inner socket's streams, we can open as many sockets as we
> like during authentication, and then just store the last one that
> connected as this internal socket? This seems like a way to "change
> socket identities" like I suggested.
> 
> Thank you
> Damjan
> 
> On Mon, Mar 1, 2010 at 4:29 PM, Christopher Hegarty - Sun Microsystems
> Ireland <Christopher.Hegarty at sun.com> wrote:
>> Hi Damjan,
>>
>> Sorry for the delayed response, I'm juggling too many things!
>>
>> I'm not sure how important Authentication is here. I initially pointed it
>> out, but hadn't done sufficient scoping so didn't encounter these issues.
>> Let me take another look at it, but I'm wondering if we should just proceed
>> with it as is for now. Authentication is the reason that this RFE never made
>> it back yet and has been sitting for a few years now.
> 
> 
>> -Chris.
>>
>> On 24/02/2010 17:28, Damjan Jovanovic wrote:
>>> On Mon, Feb 22, 2010 at 5:02 PM, Christopher Hegarty - Sun
>>> Microsystems Ireland<Christopher.Hegarty at sun.com>  wrote:
>>>> Hi Damjan,
>>>>
>>>> Actually, I did some work on this back in 2006 (!), but never finished
>>>> it. I
>>>> brought the changes into a mercurial repository and created a webrev:
>>>>
>>>>  http://cr.openjdk.java.net/~chegar/6370908/webrev.00/webrev/
>>>>
>>>> Basically, this change provides the basic functionality, without any
>>>> frills,
>>>> authentication, etc. I think for tunneling sockets through a HTTP proxy
>>>> it
>>>> should be sufficient. Do you require authentication in your environment?
>>>>
>>>> To have authentication supported we would need to restructure the HTTP
>>>> protocol handler in sun.net.www.protocol.http.HttpURLConnection, so that
>>>> we
>>>> can take advantage of the authentication schemes it already supports. Not
>>>> a
>>>> big deal, just needs to be done.
>>> The major problem with authentication is that there is no guarantee
>>> the HTTP connection will be persistent. Authentication generally
>>> requires making additional requests to the proxy, so we may need to
>>> connect to the proxy more than once. Each socket can only connect
>>> once, and the HttpConnectSocketImpl is a single immutable socket, so
>>> it cannot directly be used for authentication.
>>>
>>> A workaround may be to connect with a different socket for the initial
>>> request, and then connect through the HttpConnectSocketImpl's own
>>> socket for the final request. But then the problem becomes that if the
>>> proxy doesn't require authentication, the initial socket will succeed
>>> in connecting to the target host, and it will be too late to stop it
>>> and use HttpConnectSocketImpl's own socket...
>>>
>>> So we have the following options:
>>> 1. Only support proxies for which the entire request chain can be made
>>> over a single persistent connection.
>>> 2. Connect to each target host up to twice, once when we believe the
>>> proxy needs authentication but doesn't, and then again later with the
>>> real socket. This will probably kill protocols that use one shot
>>> connections, like FTP.
>>> 3. Find some way to change socket identities, then use as many sockets
>>> as it takes and somehow adopt the final socket into
>>> HttpConnectSocketImpl's own.
>>>
>>> Any suggestions?
>>>
>>>> -Chris.
>>> Damjan
>>>
>>>> On 21/02/2010 13:09, Damjan Jovanovic wrote:
>>>>> Hi
>>>>>
>>>>>>  From http://bugs.sun.com/view_bug.do?bug_id=6370908
>>>>> This RFE is basically about getting a TCP socket to tunnel through an
>>>>> HTTP proxy using the HTTP CONNECT request.
>>>>>
>>>>> I've found a hack to get this feature to work, using sun.net.*
>>>>> packages and lots of reflection. Would it be acceptable to use this
>>>>> solution (with some way to change socket identity) in a patch that
>>>>> adds a java.net.HttpSocketImpl class similar to the
>>>>> java.net.SocksSocketImpl class that's already used to tunnel through
>>>>> SOCKS proxies? If not, in what other way should such a patch be done?
>>>>>
>>>>> Thank you
>>>>> Damjan Jovanovic
>>>>>
>>>>> import java.net.*;
>>>>> import java.io.*;
>>>>> import java.lang.reflect.*;
>>>>>
>>>>> public class TunnelProxy {
>>>>>        private static Socket connectThroughHTTPProxy(String proxyHost,
>>>>> int
>>>>> proxyPort, String destinationHost, int destinationPort) throws
>>>>> Exception
>>>>>        {
>>>>>                URL destinationURL = new URL("http://" + destinationHost
>>>>> +
>>>>> ":" +
>>>>> destinationPort);
>>>>>                sun.net.www.protocol.http.HttpURLConnection conn =
>>>>>                        new sun.net.www.protocol.http.HttpURLConnection(
>>>>>                                destinationURL, new
>>>>> java.net.Proxy(java.net.Proxy.Type.HTTP, new
>>>>> InetSocketAddress(proxyHost, proxyPort)));
>>>>>                conn.setDoInput(true);
>>>>>                conn.setDoOutput(true);
>>>>>                conn.connect();
>>>>>                conn.doTunneling();
>>>>>                Field httpField =
>>>>> conn.getClass().getDeclaredField("http");
>>>>>                httpField.setAccessible(true);
>>>>>                sun.net.www.http.HttpClient httpClient =
>>>>> (sun.net.www.http.HttpClient) httpField.get(conn);
>>>>>                Field serverSocketField =
>>>>> sun.net.NetworkClient.class.getDeclaredField("serverSocket");
>>>>>                serverSocketField.setAccessible(true);
>>>>>                Socket socket = (Socket)
>>>>> serverSocketField.get(httpClient);
>>>>>                return socket;
>>>>>        }
>>>>>
>>>>>        public static void main(String[] args) throws Exception {
>>>>>                System.setProperty("java.net.useSystemProxies", "true");
>>>>>                InputStream in = connectThroughHTTPProxy(args[0],
>>>>> Integer.parseInt(args[1]), args[2],
>>>>> Integer.parseInt(args[3])).getInputStream();
>>>>>                byte[] bytes = new byte[1024];
>>>>>                int bytesRead;
>>>>>                while ((bytesRead = in.read(bytes)) != -1) {
>>>>>                        System.out.print(new String(bytes));
>>>>>                }
>>>>>        }
>>>>> }