NTLM failed with both USE_OPENSSL and USE_WINDOWS_SSPI enabled

[fengyue520]

I did this

Using the release version(8.7.1) of the libcurl library, I added the USE_OPENSSL and USE_WINDOWS_SSPI macros.

Many customers have reported that after updating with this EWS patch from Microsoft (link), NTLM authentication fails during HTTPS requests.

The log is as follows:
[my_curl_debug_callback] This: 4086309000 HEADER_IN :HTTP/1.1 401 Unauthorized

[my_curl_debug_callback] This: 4086309000 HEADER_IN :request-id: 4d022f2e-c9ba-4954-a4a6-ea7bdf516da4

[my_curl_debug_callback] This: 4086309000 HEADER_IN :X-OWA-Version: 15.1.2507.39

[my_curl_debug_callback] This: 4086309000 HEADER_IN :WWW-Authenticate: Negotiate

[my_curl_debug_callback] This: 4086309000 TEXT :NTLM handshake rejected

[my_curl_debug_callback] This: 4086309000 TEXT :Authentication problem. Ignoring this.

[my_curl_debug_callback] This: 4086309000 HEADER_IN :WWW-Authenticate: NTLM

[my_curl_debug_callback] This: 4086309000 HEADER_IN :X-FEServer: E16DN28C-S1E7

[my_curl_debug_callback] This: 4086309000 HEADER_IN :WWW-Authenticate: Basic realm="xxx.com"

[my_curl_debug_callback] This: 4086309000 HEADER_IN :Date: Thu, 09 May 2024 13:17:04 GMT

[my_curl_debug_callback] This: 4086309000 HEADER_IN :Content-Length: 0

[my_curl_debug_callback] This: 4086309000 HEADER_IN :

[my_curl_debug_callback] This: 4086309000 TEXT :Connection cache is full, closing the oldest one

[my_curl_debug_callback] This: 4086309000 TEXT :Closing connection

[my_curl_debug_callback] This: 4086309000 TEXT :TLSv1.3 (OUT), TLS alert, close notify (256):

[my_curl_debug_callback] This: 4086309000 TEXT :Connection #151 to host xxx.com left intact

I expected the following

NTLM auth success & https request is normal

curl/libcurl version

8.7.1

operating system

windows platform have this issue only.

[fengyue520]

A few years ago, I also reported this issue(#9579), and it was fixed at that time in my local. However, we later found that a small number of customers were still experiencing this problem. Therefore, we optimized the previous code, and customers have confirmed that this solution resolved the issue. I am preparing to submit a pull request. Before that, let me explain the root cause.

[fengyue520]

The root cause of the problem is that after customers updated the Microsoft EWS patch, a setting called "Windows Extended Protection" changed from 'disabled' by default to 'allowed'. With this setting changed, the client can choose to use a stronger NTLM authentication logic, which involves using the CBT (channel binding token) field to further ensure the security of the request.

When we compile libcurl with the USE_OPENSSL and USE_WINDOWS_SSPI macros, the requests sent from libcurl will choose to use this field. However, during verification, libcurl does not actually calculate this field, leading to NTLM authentication failure.

When I use the curl tool to send HTTPS requests directly to the NTLM server, I find that there is no issue. This is because the curl tool on the Windows platform uses Schannel & USE_WINDOWS_SSPI, which has already been fixed in issuehttps://github.com//issues/3280. The fixed MR is here https://github.com/curl/curl/pull/3321/files. We can see from this MR that when using the Schannel & USE_WINDOWS_SSPI macro, the CBT field is assigned in ### lib/vtls/schannel.c and then passed into the SSPI interface QueryContextAttributes through the SEC_CHANNEL_BINDINGS structure for further processing by the SSPI interface. When we use OpenSSL & USE_WINDOWS_SSPI, there is no place where the CBT field is assigned, resulting in its absence.

Therefore, when using OpenSSL & USE_WINDOWS_SSPI, we need to calculate the CBT field separately and then pass it into the SSPI interface.

I will submit a pull request to explain my changes, which have already been validated by the customers experiencing the issue.

[MarcelRaad]

This is the same as #13098, just for SSPI + NTLM instead of GSSAPI + Negotiate, and a fix will probably conflict with it, right?

[fengyue520]

thanks your reply, seems has conflict ,let me check & test that code. @MarcelRaad

[Foorack]

@fengyue520 Did you mean #13098 had conflicts? If so they should be resolved now, as I just rebased on master. Did you have time to test if that patch fixes your problem?