Enhance proxy support in java.net core libraries

[Jaikiran Pai]

Hello Alessandro,

I've moved this discussion to net-dev mailing list which is more 
relevant for this discussion (and Bcced core-libs-dev). If you haven't 
already subscribed to net-dev, then you can do it here 
https://mail.openjdk.org/mailman/listinfo/net-dev.

-Jaikiran

On 13/05/24 3:28 am, Alessandro Autiero wrote:
> Hello, my name is Alessandro Autiero and I'd like to propose three 
> enhancements for the java core libraries to better support proxies in 
> network components of the JDK.
>
> There are three classes in the java.net <http://java.net> package that 
> have proxy support:
>
>   * java.net.Socket
>     Introduced in Java 1.0, supports HTTP(S)/SOCKS proxies modelled by
>     java.net.Proxy through the java.net.Socket(java.net.Proxy) constructor
>   * java.net.HttpURLConnection
>     Introduced in Java 1.1, supports HTTP(S)/SOCKS proxies modelled by
>     java.net.Proxy through the
>     java.net.URL#openConnection(java.net.Proxy) public method
>   * java.net.HttpClient (Introduced in Java 11)
>     Introduced in Java 11, supports HTTP(S) proxies modelled by
>     java.net.ProxySelector through the public
>     proxy(java.net.ProxySelector) method in its builder or the default
>     java.net.ProxySelector, which can be set by calling
>     java.net.ProxySelector#setDefault(java.net.ProxySelector)
>
> While most proxies provide support for both the HTTP and SOCKS scheme, 
> considering that the older HTTP client API had support for both, 
> developers might choose to use the older api, or to use an external 
> one, if they need or want to provide support for this feature. A quick 
> Google search for a recommendation on which Http Client to use on a 
> modern Java version yields many results that list SOCKS support as a 
> feature to keep in mind when making a choice. While this is not 
> necessarily indicative of the average Java developer sentiment about 
> the feature, I think that it should be considered, alongside a couple 
> of issues that were opened on StackOverFlow 
> <https://stackoverflow.com/questions/70011046/how-to-use-a-socks-proxy-with-java-jdk-httpclient> 
> asking about support for this feature. Accordingly, I propose adding 
> support for SOCKS proxies in java.net.HttpClient. If the change is 
> allowed, consider that the default java.net.ProxySelector is an 
> instance of sun.net.spi.DefaultProxySelector, which supports SOCKS 
> proxies, but this implementation cannot be initialized by the user as 
> it's not exposed by the module system. Starting from Java 9, 
> ProxySelector#of(InetSocketAddress) was introduced, which returns an 
> instance of java.net.ProxySelector$StaticProxySelector 
> <https://github.com/openjdk/jdk/blob/5053b70a7fc67ce9b73dbeecbdd88fbc34d45e04/src/java.base/share/classes/java/net/ProxySelector.java#L194>, 
> a static inner class of ProxySelector introduced in Java 9 which only 
> implements support for HTTP(S) proxies. StaticProxySelector's 
> constructor could be modified from 
> StaticProxySelector(java.net.InetSocketAddress) to 
> StaticProxySelector(java.net.Proxy$Type, java.net.InetSocketAddress) 
> to initialize the java.net.Proxy instance with a specified proxy type 
> instead of hard coding HTTP. Then we could overload the method 
> ProxySelector#of(InetSocketAddress) with 
> ProxySelector#of(java.net.Proxy$Type, InetSocketAddress) method which 
> would invoke the constructor we defined earlier. This change would not 
> be breaking as StaticProxySelector is package private, no public 
> methods would be deleted and the default scheme would still be HTTP. 
> jdk.internal.net.http.HttpRequestImpl uses the ProxySelector in its 
> retrieveProxy method, but checks if the proxy is an HTTP proxy: this 
> would need to be changed as well. Finally, considering that unlike 
> HttpURLConnection, HttpClient doesn't delegate the underlying 
> connection to java.net.Socket, the java.net.http module would need to 
> be enhanced to support SOCKS authentication, which could take more effort.
>
> Another observation that I've made is about authentication. If a proxy 
> requires basic authentication, that is authentication through a 
> username and optionally a password, a developer can implement the 
> java.net.Authenticator class and override the 
> getPasswordAuthentication method. While basic authentication is still 
> the norm for most proxies, it's disabled by default in the JDK since 
> Java 8. Though, it's possible to enable it by overriding the net 
> properties jdk.http.auth.proxying.disabledSchemes and 
> jdk.http.auth.tunneling.disabledSchemes using System.setProperty. I 
> couldn't find an explanation about why this change was implemented, so 
> I can only speculate that it was done to incentivize Java developers 
> to use an IP whitelist instead of basic auth to improve security, 
> assuming that the connection isn't secure(HTTP). The problem though is 
> that the net properties that need to be changed to allow basic proxy 
> authentication are only read only one time in the static initializer 
> of sun.net.www.protocol.http.HttpURLConnection class, the underlying 
> implementation of java.net.HttpURLConnection. So, if for example a 
> library loads this class before the developer calls 
> System.setProperty, the change will have no effect and the 
> authentication will subsequently fail. This may seem like an edge 
> case, but for example in a Spring Boot environment, this exact issue 
> will arise if the property isn't set before calling 
> SpringApplication.run. I think that the best solution would be to 
> remove the disabledTunnelingSchemes and disabledProxyingSchemes static 
> fields from sun.net.www.protocol.http.HttpURLConnection and read the 
> net properties when they are used instead of caching them. This 
> solution is not a breaking change and should be very easy to implement 
> as both fields are only referenced when initializing a 
> AuthenticationHeader in the same class. Additionally, if we can agree 
> on the fact that basic authentication is still the predominant way to 
> provide authentication capabilities for a proxy and/or that disabling 
> it doesn't provide a direct security benefit, I propose to also set 
> jdk.http.auth.proxying.disabledSchemes and 
> jdk.http.auth.tunneling.disabledSchemes to an empty String so no 
> schemes are disabled by default. This change could be breaking for 
> Applications developed starting from Java 8 that expect basic 
> authentication to be disabled, but I think that the scope of the 
> impact would be much smaller, if there would be any at all, than when 
> these flags were introduced breaking basic authentication for existing 
> applications upgrading to Java 8.
>
> The last issue I've noticed is also about authentication. If a 
> developer wants to set an instance of Authenticator instead of relying 
> on the default one, which can be set using 
> Authenticator.setAuthenticator, this may not be possible depending on 
> the implementation:
>
>   * java.net.Socket
>     Not supported
>   * java.net.HttpURLConnection
>     Supported through the setAuthenticator(Authenticator) method
>     introduced in Java 9
>   * java.net.HttpClient
>     Supported through the authenticator(Authenticator) method in its
>     builder
>
> The reason why a developer might want to provide an instance of 
> Authenticator instead of relying on the default one is that, for 
> example, in a concurrent environment where multiple sockets exist, 
> each using a different proxy, if the proxy host and port are the same, 
> but each Socket instance is expected to use a different pair of 
> username and passwords, the default Authenticator cannot determine 
> which pair of credentials should be assigned to a given authentication 
> request as it lacks the scope to make this decision. This change is 
> not breaking as the default authenticator of a Socket would still be 
> the default authenticator, which is the current behaviour. If the 
> change is allowed, a possible solution would be to add a private field 
> named authenticator and a public method 
> setAuthenticator(Authenticator) to java.net.Socket. Then 
> HttpConnectSocketImpl, the socket implementation for a socket using an 
> http(s) proxy, would need to use the authenticator of its delegating 
> socket instead of the default one in the doTunnel method: this is a 
> one line of code change as HttpURLConnection already has support for 
> setAuthenticator from Java 9. Finally, SocksSocketImpl, the socket 
> implementation for a socket using a socks4/5 proxy, would need to use 
> the authenticator of the delegating socket in the authenticate method. 
> instead of calling Authenticator.requestPasswordAuthentication, which 
> uses the default authenticator.
>
> I am happy to work on all, if any, of the enhancements that are 
> considered feasible.
> I'm also looking forward to any possible criticism and feedback.
> Thanks in advance.
>
> I
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/net-dev/attachments/20240513/d8636596/attachment-0001.htm>