AW: ntlm with ms exchange server not working since java 1.7

Mon Oct 10 12:45:27 PDT 2011

Hi! back again ;-)

>> To my surprise Digest will be chosen first (this is documented in sun.net.www.protocol.http.AuthenticationHeader, but wrong I think, shouldn't it be read like this:  negotiate ->  kerberos ->  ntlm ->  digest ->  basic) - also no other method will 
>> be tried yet.

> This is not a surprise. In order for Negotiate to happen, you need some Kerberos settings and without them it will not go very long. If I remember correctly, Digest is preferred to NTLM so it's the natural fallback.

Just the part, that digest is preferred over NTLM was the surprising part. If you've configured an Swing-Authenticator this will prevent the application from transparently authenticate against the server, no?
And then, also that the system did not try to NTLM, after Digest failed, surprises me.

> That seems to be the only approach. But I still hope it is a server configuration error.
I've checked this against 3 different Exchange Server (2007+2010), all show the same behavior. Two of them are installed by me and one is a server from an exchange server hoster (http://itsolutionsnow.com/)
And since my iPhone works against all these 3 server and my Java 1.6 App works too, I don't think this is a server issue. We are just digging in undocumented areas here ... 

>> Instead of passing "dontUseNegotiate" to the AuthenticationHeader, we will pass in a Set of already tried authentication methods.

> I'm not sure of that. The code logic here is quite fragile and it has underdone several rounds of update and tweaking, and I surely don't like to see anything broken.
I see that the code is ... well ... evolved and highly complicated. And I perfectly understand that you would not like to change much here as it seems VERY hard to test.

> A rather ugly hack is to choose NTLM as long as it has extra parameter(s). Hopefully that's safe. We know NTLM has defined 3 messages and that means the final answer from server is simply 200 OK without any 4th confirmation...
Yep, that will fix my main issue, but will not help in trying all the possible authentication methods announced by the server.

I couldn't resist, sorry, attached you will find a diff and the source files I had to change to patch both issues (in my environment at least). I active them in an bootclasspath/p environment. Don't be too curious about AuthenticationHeader2, I just had to rename it to avoid a conflict with javas original class as I changed the constructor.
As you can see, the patch is not that big. Which does not mean it doesn't breaks something ... I'll go with them in my test environment for some days.
I would be more than happy if you could give it an hour or two to have a look at it. Unhappily, I am sure you might need more time to test all use-cases, but I am still hoping ... ;-)
Also, if we'd like to keep Digest before NTLM, I'd vote for adding a property to change the ordering of the authentication methods. But this means have you accept my patch first, else the fallback does not work "as expected" anyway.

Well, probably we can (if required, and that is for sure) bugfix my patch and have fixed two issues with it then. ;-)
Thanks for your time!

Ciao,
Mario

> This also should make it possible to avoid the use of inNegotiate.
>
> What do you think?
> I think I am going to play a bit with this idea ... :-)
>
> Ciao,
> Mario
>
> [1] http://www.ietf.org/rfc/rfc2616.txt
> [2] http://msdn.microsoft.com/en-us/library/aa479391.aspx
>
> -----Ursprüngliche Nachricht-----
> Von: Weijun Wang [mailto:weijun.wang at oracle.com]
> Gesendet: Montag, 10. Oktober 2011 17:33
> An: Chris Hegarty
> Cc: Mario Ivankovits; net-dev at openjdk.java.net
> Betreff: Re: ntlm with ms exchange server not working since java 1.7
>
> During an NTLM handshake, I've never seen a server mentioning another scheme. As seen in message #4, the NTLM header still contains data, so there should not be WWW-Authenticate: Negotiate header.
>
> That said, this is only my experience. I tried to find any words on this from an RFC but failed.
>
> -Max
>
>
>
> On Oct 10, 2011, at 7:48 AM, Chris Hegarty<chris.hegarty at oracle.com>  wrote:
>
>> Max [to'ed],
>>
>> Does this look familiar? Is it wrong for the server to be returning "WWW-Authenticate: Negotiate" during NTLM handshake?
>>
>> -Chris.
>>
>> On 08/10/2011 14:41, Mario Ivankovits wrote:
>>> Hi net-devs,
>>>
>>> I hope you do not mind that I post to this list, but I hope I can 
>>> provide enough in-depth information about the problem to justify the 
>>> post here.
>>>
>>> Accessing a “normal” ntlm protected resource – a simple index.html 
>>> in an protected directory on an IIS 7.5 server - the ntlm 
>>> authentication works fine.
>>>
>>> However, trying to access the Microsoft Exchange 2010 webservice 
>>> failes with “401 Unauthorized”.
>>>
>>> I used this few lines to debug the connection/authentication process
>>>
>>> URL url = new URL("https://exchange/ews/Services.wsdl");
>>>
>>> byte[] buf = new byte[10240];
>>>
>>> int read = url.openStream().read(buf);
>>>
>>> System.err.println(new String(buf, 0, read));
>>>
>>> This snipped works fine in java 1.6, but failes with an IOException 
>>> (http status 401) in java 1.7.
>>>
>>> I found an interesting difference when accessing the “normal”
>>> web-page and the exchange webservice.
>>>
>>> When accessing the web-page, the server answers “WWW-Authenticate:
>>> Negotiate” just after the first 401 response which triggers the 
>>> authentication process then. In contrast, when accessing the 
>>> Exchange webservice the “WWW-Authenticate: Negotiate” is sent during 
>>> the negotiation process too, which then triggers the inNegotiate 
>>> flag in sun.net.www.protocol.http.HttpURLConnection in 
>>> getInputStream and let the negotiation process fail.
>>>
>>> If I hack the response values and change any subsequent Negotiate to 
>>> e.g. NegotiateXX, then the inNegotiate flag will not change and the 
>>> authentication process will finish and authentication finally works.
>>>
>>> Here is the request/response cycle which fail then:
>>>
>>> #1: {GET /ews/Services.wsdl HTTP/1.1: null}{User-Agent:
>>> Java/1.7.0_02-ea}{Host: exchange }{Accept: text/html, image/gif, 
>>> image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}
>>>
>>> #2: {null: HTTP/1.1 401 Unauthorized}{Server:
>>> Microsoft-IIS/7.5}{WWW-Authenticate: Negotiate}{WWW-Authenticate:
>>> NTLM}{X-Powered-By: ASP.NET}{Date: Sat, 08 Oct 2011 13:17:39
>>> GMT}{Content-Length: 0}
>>>
>>> #3: {GET /ews/Services.wsdl HTTP/1.1: null}{User-Agent:
>>> Java/1.7.0_02-ea}{Host: exchange }{Accept: text/html, image/gif, 
>>> image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}{Authorization:
>>> NTLM MY_NTLM_DATA}
>>>
>>> #4: {null: HTTP/1.1 401 Unauthorized}{Server:
>>> Microsoft-IIS/7.5}{WWW-Authenticate: NTLM
>>> SERVER_NTLM_DATA}{WWW-Authenticate: Negotiate}{X-Powered-By:
>>> ASP.NET}{Date: Sat, 08 Oct 2011 13:17:39 GMT}{Content-Length: 0}
>>>
>>> Exception in thread "main" java.io.IOException: Server returned HTTP 
>>> response code: 401 for URL: https://exchange/ews/Services.wsdl
>>>
>>> at
>>> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLCo
>>> n
>>> nection.java:1612)
>>>
>>> at
>>> sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Htt
>>> p
>>> sURLConnectionImpl.java:254)
>>>
>>> at java.net.URL.openStream(URL.java:1035)
>>>
>>> Does this make sense to you?
>>>
>>> It seems to me the “inNegotiate” handling needs a review as it does 
>>> not work in all cases.
>>>
>>> I hope my informations are of any help to fix this issue.
>>>
>>> Ciao,
>>>
>>> Mario
>>>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: AuthenticationHeader.patch
Type: application/octet-stream
Size: 2393 bytes
Desc: AuthenticationHeader.patch
Url : http://mail.openjdk.java.net/pipermail/net-dev/attachments/20111010/2e25e13e/AuthenticationHeader.patch 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: HttpURLConnection.patch
Type: application/octet-stream
Size: 3388 bytes
Desc: HttpURLConnection.patch
Url : http://mail.openjdk.java.net/pipermail/net-dev/attachments/20111010/2e25e13e/HttpURLConnection.patch 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: AuthenticationHeader2.java
Type: application/octet-stream
Size: 10630 bytes
Desc: AuthenticationHeader2.java
Url : http://mail.openjdk.java.net/pipermail/net-dev/attachments/20111010/2e25e13e/AuthenticationHeader2.java 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: HttpURLConnection.java
Type: application/octet-stream
Size: 132996 bytes
Desc: HttpURLConnection.java
Url : http://mail.openjdk.java.net/pipermail/net-dev/attachments/20111010/2e25e13e/HttpURLConnection.java 