invalid peer certificate: BadSignature when installing package from private index using ECDSA SHA-512 SSL cert

[kcon-stackav]

I use a private Python package index server (anonymized as pypi.internal in the snippets below) that uses an ecdsa-with-SHA512 SSL certificate:

❯ openssl s_client -connect pypi.internal:443 < /dev/null 2>/dev/null | openssl x509 -in /dev/stdin -text -noout -certopt ca_default -certopt no_validity -certopt no_serial -certopt no_subject -certopt no_extensions  -certopt no_sigdump
        Signature Algorithm: ecdsa-with-SHA512

I am able to use pip to fetch and install Python packages from this index, but when I try to use uv it fails with the following error:

❯ uv pip install https://pypi.internal/pypi/package/version/package-version-py3-none-any.whl
error: Failed to download: `package @ https://pypi.internal/pypi/package/version/package-version-py3-none-any.whl`
  Caused by: error sending request for url (https://pypi.internal/pypi/package/version/package-version-py3-none-any.whl)
  Caused by: client error (Connect)
  Caused by: invalid peer certificate: BadSignature

❯ uname -mv
#20~22.04.1-Ubuntu SMP Wed May  1 16:10:50 UTC 2024 x86_64

❯ uv --version
uv 0.2.15

I think this may be the same issue described by rust-lang/rustup#3820, because I understand uv depends on reqwest:

uv/Cargo.toml

Line 112 in c28a2c7

reqwest = { version = "0.12.3", default-features = false, features = ["json", "gzip", "brotli", "stream", "rustls-tls", "rustls-tls-native-roots"] }

which depends on rustls using the ring feature: https://github.com/seanmonstar/reqwest/blob/c4ebb073438026e09c99469be02fc1f1a254058a/Cargo.toml#L181

and ring does not yet support the ECDSA SHA-512 certificate signature algorithm (WIP but has been open for over 8 months so it's not clear when it would land and be released: briansmith/ring#1631).

I was reading here that it may be possible to configure reqwest to use aws-lc-rs (which does provide support for the ECDSA SHA-512 algorithm for rustls) instead of ring. Here is a draft PR where it looks like rustup is trying to take this approach: rust-lang/rustup#3898

Would ya'll consider switching uv from using ring to aws-lc-rs to support fetching Python packages from an index server whose SSL certificate uses the ECDSA SHA-512 signature algorithm?

[samypr100]

Could you share your private package index TLS setup to reproduce? Are you using nginx as a reverse proxy?

[kcon-stackav]

@samypr100 thank you for the quick response. 🙏 My private package index's SSL certificate is self-signed by a private CA, and yes I'm using nginx as a reverse proxy where TLS is terminated at the proxy.

[samypr100]

Thanks for confirming @kcon-stackav. Would you be able to share some of the tls settings of the nginx config you're using? (e.g. ssl_protocols, ssl_ciphers, etc.) and also the openssl command used to generate the self-signed certificate? It would help in reproducing the issue.

[zanieb]

Thanks for the thorough write-up and cross references, that's really helpful. I'd be willing to consider changing backends, though I worry it'd be a breaking change and don't fully understand the implications yet. Would you be interested in opening a pull request to explore the change?

[Sovietaced]

Thanks for confirming @kcon-stackav. Would you be able to share some of the tls settings of the nginx config you're using? (e.g. ssl_protocols, ssl_ciphers, etc.) and also the openssl command used to generate the self-signed certificate? It would help in reproducing the issue.

We are using nginx with certificates managed by cert-manager. Cert-manager is hooked up to AWS private CA as an issuer. Certificate generation is automated for us. Presumably if you use a ECDSA private key with enough bits it will sign the cert with the problematic algorithm.

[kcon-stackav]

Thanks for the thorough write-up and cross references, that's really helpful. I'd be willing to consider changing backends, though I worry it'd be a breaking change and don't fully understand the implications yet. Would you be interested in opening a pull request to explore the change?

Thank you! Sure I'll work on a pull request to try to make the change. 🙏

[kcon-stackav]

I created this PR to try out using aws-lc-rs instead of ring for TLS: #4734

Unfortunately it looks like it won't solve our issue 😞. Although aws-lc-rs supports the ECDSA SHA-512 certificate signature algorithm we need, our certificate trust chain's issuing CA uses ECDSA P384 for its subject public key algorithm, and I learned that aws-lc-rs does not support verifying that particular combination of subject public key algorithm and signature algorithm.

Instead it only supports verifying a ECDSA SHA-512 signature when the subject public key algorithm is ECDSA P521:

https://github.com/rustls/webpki/blob/dba3b84bcf810612a83c8fd3c750d3c4106e7b9e/src/aws_lc_rs_algs.rs#L70-L75

Or otherwise verifying a ECDSA SHA-384 signature when the subject public key algorithm is ECDSA P384:

https://github.com/rustls/webpki/blob/dba3b84bcf810612a83c8fd3c750d3c4106e7b9e/src/aws_lc_rs_algs.rs#L63-L68

I'm not sure changing our certs to match one of those combinations will be an option, so now I'm thinking our best option may be to leverage #1339 once it is available.

[cpu]

Unfortunately it looks like it won't solve our issue 😞. Although aws-lc-rs supports the ECDSA SHA-512 certificate signature algorithm we need, our certificate trust chain's issuing CA uses ECDSA P384 for its subject public key algorithm, and I learned that aws-lc-rs does not support verifying that particular combination of subject public key algorithm and signature algorithm.

@kcon-stackav Have you tried with the recently released rustls 0.23.12 & the aws-lc-rs provider? There have been a handful of related PRs across the ecosystem:

• Support P521 SHA-256/SHA-384 signatures with aws-lc-rs  rustls/rustls#2050
• Add support for P-521 curve with SHA-256/SHA-384 digests rustls/webpki#272
• ECDSA P-521 with SHA1, SHA224, SHA256, SHA384 aws/aws-lc-rs#461