6370908: Add support for HTTP_CONNECT proxy in Socket class
Christopher Hegarty - Sun Microsystems Ireland
Christopher.Hegarty at Sun.COM
Fri Mar 12 06:18:25 PST 2010
Hi Damjan,
I've come full circle on this and created a new webrev based on your
original proposal. Using the current HTTP protocol handler that already
implements the tunneling and authentication. See the updated webrev:
http://cr.openjdk.java.net/~chegar/6370908/webrev.01/
I've used "more" reflection to avoid creating static dependencies on the
internal http implementation. The addition of a test will ensure that
any changes to the implementation will be picked up very quickly, i.e.
the test will break.
-Chris.
On 04/03/2010 11:29, Damjan Jovanovic wrote:
> Hi Chris
>
> Proxies are on the application layer, TCP is a transport layer
> protocol below. Using the application layer to establish in-band
> transport layer connections is dodgy by its very nature. The multiple
> sockets to get through an HTTP proxy issue is not unique to Java.
> Here's a quick look at how several open source applications have dealt
> with the problem:
>
> * prtunnel (http://joshbeam.com/software/prtunnel.php), a C
> application for tunneling through several proxy types, only supports
> no authentication and BASIC HTTP authentication. proxychains does the
> same. XChat does the same.
> * ntlmaps.sourceforge.net and similar apps work by acting as another
> proxy themselves. They can use as many sockets as necessary on the
> outgoing end to the second proxy.
> * pidgin (www.pidgin.im), an instant messaging app, has very good
> proxy support including NTLM authentication for HTTP proxies. It only
> uses a single socket and requires the proxy to use a persistent
> connection. proxytunnel (proxytunnel.sourceforge.net) does the same.
>
> I like pidgin's solution the best, and since it even works with NTLM,
> the other authentication types should be easy. Most HTTP 1.1 proxies
> should support persistent connections - after all, that feature
> benefits proxies the most. So we should be able to get away with one
> socket in most cases, maybe even for all commonly used HTTP proxies.
> I'll try to do some tests this weekend.
>
> Also, nothing stops a Socket from changing its impl. We would need a
> very clever impl that somehow remembers options that have been set and
> reapplies them to the new impl, but there's no theoretical reason why
> connect() cannot create as many impls as are necessary to get through
> the proxy.
>
> On the other hand, all Java's solutions for connecting through proxies
> are poor: SOCKS 4 is broken
> (http://mail.openjdk.java.net/pipermail/net-dev/2010-February/001580.html),
> server sockets are unsupported, datagram sockets are unsupported, and
> all java.nio channels are unsupported. But then proxies aren't so
> common any more: ADSL routers typically use NAT instead, and we don't
> provide any NAT traversal support for server sockets and datagram
> sockets using UPnP's IGD or Bonjour's NAT-PMP protocols. Maybe we
> should?
>
> Damjan
>
> On Wed, Mar 3, 2010 at 3:55 PM, Christopher Hegarty -Sun Microsystems
> Ireland<Christopher.Hegarty at sun.com> wrote:
>> Hi Damjan,
>>
>> After spending more time looking at this, I still haven't found an elegant
>> solution for supporting authentication.
>>
>> You've correctly identified an issue with the fact we have no guaranteed of
>> a persistent connection with the proxy. Changing the socket identity is not
>> as simple as overriding getIn/OutputStream since other methods like
>> setting/getting options, close, shutdown, etc use the impls fd field. This
>> field needs to refer to the correct file descriptor object.
>>
>> There is quite a lot of logic in the http protocol handler to support
>> authentication and looking at it again it is closely tied to the handler
>> implementation itself. I would prefer to reuse this code if possible.
>>
>> Have you made any progress on this? If we want authentication it may be more
>> straight forward to revert back to your original proposal of using
>> HttpURLConnection directly and having its socket exposed. Though, I'm still
>> not convinced of this solution either.
>>
>> -Chris
>>
>> Damjan Jovanovic wrote:
>>>
>>> Hi Chris
>>>
>>> I think that without authentication, you'll just get another RFE
>>> asking for authentication. Many corporate proxies require
>>> authentication, especially for a liberal request like CONNECT.
>>>
>>> java.net.Socket seems to delegate its impl field's methods. If we have
>>> another, internal socket in HttpConnectSocketImpl, and we override
>>> getInputStream() and getOutputStream() in HttpConnectSocketImpl to
>>> return this inner socket's streams, we can open as many sockets as we
>>> like during authentication, and then just store the last one that
>>> connected as this internal socket? This seems like a way to "change
>>> socket identities" like I suggested.
>>>
>>> Thank you
>>> Damjan
>>>
>>> On Mon, Mar 1, 2010 at 4:29 PM, Christopher Hegarty - Sun Microsystems
>>> Ireland<Christopher.Hegarty at sun.com> wrote:
>>>>
>>>> Hi Damjan,
>>>>
>>>> Sorry for the delayed response, I'm juggling too many things!
>>>>
>>>> I'm not sure how important Authentication is here. I initially pointed it
>>>> out, but hadn't done sufficient scoping so didn't encounter these issues.
>>>> Let me take another look at it, but I'm wondering if we should just
>>>> proceed
>>>> with it as is for now. Authentication is the reason that this RFE never
>>>> made
>>>> it back yet and has been sitting for a few years now.
>>>
>>>
>>>> -Chris.
>>>>
>>>> On 24/02/2010 17:28, Damjan Jovanovic wrote:
>>>>>
>>>>> On Mon, Feb 22, 2010 at 5:02 PM, Christopher Hegarty - Sun
>>>>> Microsystems Ireland<Christopher.Hegarty at sun.com> wrote:
>>>>>>
>>>>>> Hi Damjan,
>>>>>>
>>>>>> Actually, I did some work on this back in 2006 (!), but never finished
>>>>>> it. I
>>>>>> brought the changes into a mercurial repository and created a webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~chegar/6370908/webrev.00/webrev/
>>>>>>
>>>>>> Basically, this change provides the basic functionality, without any
>>>>>> frills,
>>>>>> authentication, etc. I think for tunneling sockets through a HTTP proxy
>>>>>> it
>>>>>> should be sufficient. Do you require authentication in your
>>>>>> environment?
>>>>>>
>>>>>> To have authentication supported we would need to restructure the HTTP
>>>>>> protocol handler in sun.net.www.protocol.http.HttpURLConnection, so
>>>>>> that
>>>>>> we
>>>>>> can take advantage of the authentication schemes it already supports.
>>>>>> Not
>>>>>> a
>>>>>> big deal, just needs to be done.
>>>>>
>>>>> The major problem with authentication is that there is no guarantee
>>>>> the HTTP connection will be persistent. Authentication generally
>>>>> requires making additional requests to the proxy, so we may need to
>>>>> connect to the proxy more than once. Each socket can only connect
>>>>> once, and the HttpConnectSocketImpl is a single immutable socket, so
>>>>> it cannot directly be used for authentication.
>>>>>
>>>>> A workaround may be to connect with a different socket for the initial
>>>>> request, and then connect through the HttpConnectSocketImpl's own
>>>>> socket for the final request. But then the problem becomes that if the
>>>>> proxy doesn't require authentication, the initial socket will succeed
>>>>> in connecting to the target host, and it will be too late to stop it
>>>>> and use HttpConnectSocketImpl's own socket...
>>>>>
>>>>> So we have the following options:
>>>>> 1. Only support proxies for which the entire request chain can be made
>>>>> over a single persistent connection.
>>>>> 2. Connect to each target host up to twice, once when we believe the
>>>>> proxy needs authentication but doesn't, and then again later with the
>>>>> real socket. This will probably kill protocols that use one shot
>>>>> connections, like FTP.
>>>>> 3. Find some way to change socket identities, then use as many sockets
>>>>> as it takes and somehow adopt the final socket into
>>>>> HttpConnectSocketImpl's own.
>>>>>
>>>>> Any suggestions?
>>>>>
>>>>>> -Chris.
>>>>>
>>>>> Damjan
>>>>>
>>>>>> On 21/02/2010 13:09, Damjan Jovanovic wrote:
>>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>>> From http://bugs.sun.com/view_bug.do?bug_id=6370908
>>>>>>>
>>>>>>> This RFE is basically about getting a TCP socket to tunnel through an
>>>>>>> HTTP proxy using the HTTP CONNECT request.
>>>>>>>
>>>>>>> I've found a hack to get this feature to work, using sun.net.*
>>>>>>> packages and lots of reflection. Would it be acceptable to use this
>>>>>>> solution (with some way to change socket identity) in a patch that
>>>>>>> adds a java.net.HttpSocketImpl class similar to the
>>>>>>> java.net.SocksSocketImpl class that's already used to tunnel through
>>>>>>> SOCKS proxies? If not, in what other way should such a patch be done?
>>>>>>>
>>>>>>> Thank you
>>>>>>> Damjan Jovanovic
>>>>>>>
>>>>>>> import java.net.*;
>>>>>>> import java.io.*;
>>>>>>> import java.lang.reflect.*;
>>>>>>>
>>>>>>> public class TunnelProxy {
>>>>>>> private static Socket connectThroughHTTPProxy(String proxyHost,
>>>>>>> int
>>>>>>> proxyPort, String destinationHost, int destinationPort) throws
>>>>>>> Exception
>>>>>>> {
>>>>>>> URL destinationURL = new URL("http://" + destinationHost
>>>>>>> +
>>>>>>> ":" +
>>>>>>> destinationPort);
>>>>>>> sun.net.www.protocol.http.HttpURLConnection conn =
>>>>>>> new sun.net.www.protocol.http.HttpURLConnection(
>>>>>>> destinationURL, new
>>>>>>> java.net.Proxy(java.net.Proxy.Type.HTTP, new
>>>>>>> InetSocketAddress(proxyHost, proxyPort)));
>>>>>>> conn.setDoInput(true);
>>>>>>> conn.setDoOutput(true);
>>>>>>> conn.connect();
>>>>>>> conn.doTunneling();
>>>>>>> Field httpField =
>>>>>>> conn.getClass().getDeclaredField("http");
>>>>>>> httpField.setAccessible(true);
>>>>>>> sun.net.www.http.HttpClient httpClient =
>>>>>>> (sun.net.www.http.HttpClient) httpField.get(conn);
>>>>>>> Field serverSocketField =
>>>>>>> sun.net.NetworkClient.class.getDeclaredField("serverSocket");
>>>>>>> serverSocketField.setAccessible(true);
>>>>>>> Socket socket = (Socket)
>>>>>>> serverSocketField.get(httpClient);
>>>>>>> return socket;
>>>>>>> }
>>>>>>>
>>>>>>> public static void main(String[] args) throws Exception {
>>>>>>> System.setProperty("java.net.useSystemProxies", "true");
>>>>>>> InputStream in = connectThroughHTTPProxy(args[0],
>>>>>>> Integer.parseInt(args[1]), args[2],
>>>>>>> Integer.parseInt(args[3])).getInputStream();
>>>>>>> byte[] bytes = new byte[1024];
>>>>>>> int bytesRead;
>>>>>>> while ((bytesRead = in.read(bytes)) != -1) {
>>>>>>> System.out.print(new String(bytes));
>>>>>>> }
>>>>>>> }
>>>>>>> }
>>
More information about the net-dev
mailing list