Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error trying to connect: tls handshake eof #6427

Closed
enjikaka opened this issue Jun 22, 2020 · 23 comments
Closed

error trying to connect: tls handshake eof #6427

enjikaka opened this issue Jun 22, 2020 · 23 comments
Labels
bug Something isn't working correctly cli related to cli/ dir tls Issues related to TLS implementation

Comments

@enjikaka
Copy link

enjikaka commented Jun 22, 2020

For some reason doing a GET on https://copernicus.discomap.eea.europa.eu/arcgis/rest/services/Corine/CLC2018_WM/MapServer/0?f=json
crashes Deno with:

error: Uncaught Http: error sending request for url (https://copernicus.discomap.eea.europa.eu/arcgis/rest/services/Corine/CLC2018_WM/MapServer/0?f=json): 
error trying to connect: tls handshake eof

Works fine with curl and client side JS. Something wrong with SSL management in Deno? Calling on HTTP doesn't work either since Deno upgrades to HTTPS.

Steps to reproduce

Source:

const layerInfoURL = 'https://copernicus.discomap.eea.europa.eu/arcgis/rest/services/Corine/CLC2018_WM/MapServer/0?f=json';
const json = await fetch(layerInfoURL).then(r => r.json());

console.log(json);

or

deno run --allow-net https://deno.land/x/gh:enjikaka:terrain-server/poor_api.ts

Version

deno 1.1.1
v8 8.5.104
typescript 3.9.2
macOS 10.15.5 (19F101)
also fails in docker on hayd/alpine-deno:1.1.1

@zinthose
Copy link

I'm having the same issue. ATM, I'm dead in the water as this is a blocking issue for me to adopt Deno.

I have a POST request to a remote REST API that uses a standard GoDaddy Cert so it's not self signed like I'm seeing in other issues. From my understanding the issue's root is with the TLS method used but I'm unclear how to change or correct it.

I can successfully make the call using Node or GO. The Go code used for reference is below with certain details changed due to security and confidentiality issues.

package main

import (
	"fmt"
	"io/ioutil"
	"net/http"
	"strings"
)

func main() {

	url := "https://api.test.com/webapi/api/session"
	method := "POST"

	payload := strings.NewReader("{\"UserId\":\"test@test.com\",\"Password\":\"P@$$w0rd\",\"SourceSystem\":\"BUDDY\"}")

	client := &http.Client{}
	req, err := http.NewRequest(method, url, payload)

	if err != nil {
		fmt.Println(err)
	}
	req.Header.Add("Content-Type", "application/json;charset=UTF-8")

	res, err := client.Do(req)
	defer res.Body.Close()
	body, err := ioutil.ReadAll(res.Body)

	fmt.Println(string(body))
}

@enjikaka
Copy link
Author

enjikaka commented Jun 25, 2020

I temporarily solved it on my end by proxying through another cert. https://github.com/Freeboard/thingproxy

@ry ry added bug Something isn't working correctly cli related to cli/ dir labels Jun 25, 2020
@zinthose
Copy link

I did try adding cors and origin to my request headers using soxa but to no effect.
The response headers from the server I'm attempting to connect to is below if some insight can be gleamed from it that I'm not seeing:

Access-Control-Allow-Origin: *
Cache-Control: no-cache
Content-Length: 461
Content-Type: application/json; charset=utf-8
Date: Thu, 25 Jun 2020 12:52:13 GMT
Expires: -1
Pragma: no-cache
Server: Microsoft-IIS/8.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET

@enjikaka I'll look into thingproxy as a temporary solution.

@enjikaka
Copy link
Author

@zinthose CORS doesn't matter here. It's the HTTPS management in Deno and/or the rust lib it uses that doesn't handle specific certs correctly. With that thingproxy it'll use another cert that Deno doesn't error on, which is a temporary hack around the issue. :)

@enjikaka
Copy link
Author

To compare here's curl -v with the plain server vs proxied response:

curl -v https://copernicus.discomap.eea.europa.eu/arcgis/rest/services/Corine/CLC2018_WM/MapServer/0\?f\=json

*   Trying 87.54.7.100...
* TCP_NODELAY set
* Connected to copernicus.discomap.eea.europa.eu (87.54.7.100) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / AES128-SHA
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=*.discomap.eea.europa.eu
*  start date: Apr 20 00:00:00 2020 GMT
*  expire date: Jul 23 00:00:00 2022 GMT
*  subjectAltName: host "copernicus.discomap.eea.europa.eu" matched cert's "*.discomap.eea.europa.eu"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
*  SSL certificate verify ok.
> GET /arcgis/rest/services/Corine/CLC2018_WM/MapServer/0?f=json HTTP/1.1
> Host: copernicus.discomap.eea.europa.eu
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Connection: Keep-Alive
< Content-Length: 14388
< Date: Thu, 25 Jun 2020 13:25:46 GMT
< Content-Type: application/json;charset=UTF-8
< ETag: 34ce7b3
< Server: Microsoft-IIS/10.0
< Server:
< Cache-Control: private, must-revalidate, max-age=0
< Vary: Origin
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< X-AspNet-Version: 4.0.30319
< X-Powered-By: ASP.NET
<

curl -v https://thingproxy.freeboard.io/fetch/https://copernicus.discomap.eea.europa.eu/arcgis/rest/services/Corine/CLC2018_WM/MapServer/0/\?f\=json

*   Trying 52.5.163.236...
* TCP_NODELAY set
* Connected to thingproxy.freeboard.io (52.5.163.236) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=*.freeboard.io
*  start date: Jun 15 00:00:00 2020 GMT
*  expire date: Jul 15 12:00:00 2021 GMT
*  subjectAltName: host "thingproxy.freeboard.io" matched cert's "*.freeboard.io"
*  issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7feb5e00e800)
> GET /fetch/https://copernicus.discomap.eea.europa.eu/arcgis/rest/services/Corine/CLC2018_WM/MapServer/0/?f=json HTTP/2
> Host: thingproxy.freeboard.io
> User-Agent: curl/7.64.1
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 200
< content-type: application/json;charset=UTF-8
< content-length: 14388
< x-aspnet-version: 4.0.30319
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
< vary: origin
< cache-control: private, must-revalidate, max-age=0
< server: Microsoft-IIS/10.0
< etag: 34ce7b3
< date: Thu, 25 Jun 2020 13:29:03 GMT
<

I think ALPN, server did not agree to a protocol is the issue here?

@zinthose
Copy link

I did a DIFF compare with the results below. It dose appear the protocol is the issue.

image

@zinthose
Copy link

FYI: I created a thingproxy docker image for the time being as an alternative workaround for others.

@zinthose
Copy link

I know it seems silly to put so much effort into a workaround, but here you go anyway. 🤪

I created a module, zinthose/thingproxy-deno , that can replace the fetch api to automatically forward requests through a thingproxy server.

⚠️ The module is fairly simple but may still have bugs. ⚠️

Example

const response = await phetch("https://postman-echo.com/get?foo1=bar1&oo2=bar2"); 
console.log(response.status); 
// e.g. 200 console.log(response.statusText); 
// e.g. "OK" const jsonData = await response.json();

@jacobgc
Copy link
Contributor

jacobgc commented Jul 2, 2020

Looking at the rustls library page, it looks to only support AESGCM not AES.

The europa.eu site only supports weak ciphers that rustls doesn't support.
image

Not really sure where to go from here / suggest. @ry got any ideas?

@enjikaka
Copy link
Author

enjikaka commented Jul 2, 2020

It's quite a showstopper for using Deno if you can't consume certain third-party APIs... Will be bad for adoption.

Is rustls planning support? Otherwise I guess an alternative needs to be evaluated.

@jacobgc
Copy link
Contributor

jacobgc commented Jul 2, 2020

I agree completely. In an ideal world though these sites would upgrade their SSL certs to more modern ciphers. However this isn't going to happen. I dont think rustls has any plans to add in AES without GCM however I'll raise an issue and see what I get back.

@jacobgc
Copy link
Contributor

jacobgc commented Jul 2, 2020

Got a reply at rustls/rustls#381 The host runs an old version of IIS and thus has old certificates that just aren't supported. Not sure what else to suggest. They need to move with the times. For now, I'd suggest keeping with the proxy setup you've got going.

@Spoonbender
Copy link
Contributor

As long as Deno uses rustls as its TLS library exclusively, it inherits the constraints imposed by rustls. Rustls is quite opinionated about security.

An alternative would be to enable Deno to use another TLS crate, like native-tls (that uses schannel on Windows and OpenSSL on Linux), which supports a very wide range of ciphersuites and certificates and protocols - including those that have known vulnerabilities and weaknesses. Perhaps with some "feel bad" command line flag, like deno run --allow-weak-and-broken-ciphersuites.
Then again, since Deno is "secure by default", maybe Deno should be opinionated against supporting such connections, and this issue should be closed as "won't fix".

@jacobgc
Copy link
Contributor

jacobgc commented Jul 6, 2020

The connections are still secure, just not as secure as more modern cipher suites. I do like the idea of using a wider supported SSL library like native-tls. Not sure exactly if you could block specific cipher suites though.

@Spoonbender
Copy link
Contributor

@jacobgc the native-tls crate won't let us control the exact ciphersuites, but it does enable controlling the min and max TLS protocol version, trusted root certificate, and whether to accept or reject invalid certificates.

@jacobgc
Copy link
Contributor

jacobgc commented Jul 6, 2020

@Spoonbender Certainly a much better solution than what Deno currently has.

@bartlomieju bartlomieju added the tls Issues related to TLS implementation label Oct 14, 2020
@Nexxkinn
Copy link

Hello, any progress over this case?

@lucacasonato
Copy link
Member

No.

@tsingwong
Copy link

Hey. I have the same problem , any progress over this case 😭

@pierre-luc
Copy link

I have this problem too 😭

@lucacasonato
Copy link
Member

The original issue (fetching https://copernicus.discomap.eea.europa.eu/arcgis/rest/services/Corine/CLC2018_WM/MapServer/0?f=json) is resolved. This issue is thus closed.

For others: This is not a Deno issue. Your server is using an old insecure TLS version that Deno refuses to accept. Try with curl -v to see which TLS version and cipher suite is being used. If you encounter this issue with a modern up to date cipher, please open another issue.

@acrodrig
Copy link

acrodrig commented Jul 5, 2022

@lucacasonato you answer here:

For others: This is not a Deno issue. Your server is using an old insecure TLS version that Deno refuses to accept. Try with curl -v to see which TLS version and cipher suite is being used. If you encounter this issue with a modern up to date cipher, please open another issue.

does not resolve the issue. TLS gets upgraded to better more secure versions over time. Old servers (that Deno users do not control) can keep using old versions. The old versions are not "insecure", they are less secure, as you can never achieve 100% security.

If Deno (or the underlying Rust library) is not willing to provide a way to communicate with these (admittedly) old servers, then a workaround would be highly desirable (with the appropriate warnings).

In my case, I am trying to communicate with iCloud webdav calendar servers. I doubt I will be able to convince Apple in a reasonable timeframe that their servers are "insecure" and that they should upgrade.

Don't mean to sound snarky, just pointing to a real world issue. Thanks for all the great work you've all put into Deno.

@ngdangtu-vn
Copy link

I only need a simple server for localhost. Do I need to go through everything just to do a small localhost that only live for a few hours?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working correctly cli related to cli/ dir tls Issues related to TLS implementation
Projects
None yet
Development

No branches or pull requests