no_proxy variable requires full domain names when multiple are specified

[mdegel]

Problem

We have a company network that mirrors the rust registry internally (on artifactory). Also a company proxy is in place for connections outside of the company network. Thus this proxy is configured in all the common places.

The cargo mirror is available on artifactory.company.net
The proxy is defined via common variables, e.g. https_proxy.

We noticed that the no_proxy is only respected in certain situations:

# Specifying a single generic  exclude on the domain works
export no_proxy=".company.net"

# Specifying multiple generic domain excludes do not work
export no_proxy=".company.net,.company.com"

# When specifying multiple generic domain excludes and a full domain name exclude for the server in question also works
export no_proxy=".company.net,.company.com,artifactory.company.net" 

In case of the non-working option we're stuck with errors like this:

$ cargo build
    Updating `artifactory` index
warning: spurious network error (2 tries remaining): [56] Failure when receiving data from the peer (Received HTTP code 502 from proxy after CONNECT)
warning: spurious network error (1 tries remaining): [56] Failure when receiving data from the peer (Received HTTP code 502 from proxy after CONNECT)
error: failed to get `axum` as a dependency of package `foo v0.1.0 (/home/mdegel/git/rust-cargo)`

Caused by:
  failed to query replaced source registry `crates-io`

Caused by:
  download of config.json failed

Caused by:
  failed to download from `https://artifactory.company.net/artifactory/api/cargo/crates.io-cargo-remote/index/config.json`

Caused by:
  [56] Failure when receiving data from the peer (Received HTTP code 502 from proxy after CONNECT)

With specifying the domain name completely artifactory.company.net vs .company.net we have a solution, but it's unclear to us, why this behavior differs from other tools.

Steps

No response

Possible Solution(s)

No response

Notes

Also we're using the nightly toolchain with sparse-registry.

Version

cargo 1.67.0-nightly (e027c4b5d 2022-11-25)
release: 1.67.0-nightly
commit-hash: e027c4b5d25af2119b1956fac42863b9b3242744
commit-date: 2022-11-25
host: x86_64-unknown-linux-gnu
libgit2: 1.5.0 (sys:0.15.0 vendored)
libcurl: 7.86.0-DEV (sys:0.4.59+curl-7.86.0 vendored ssl:OpenSSL/1.1.1q)
os: Ubuntu 20.04 (focal) [64-bit]

[jasal82]

I tried a bit more and found out that it has nothing to do with the full domain name. As soon as NO_PROXY contains more than one (comma-separated) entry, only the last one seems to be taken into account.

[mdegel]

@jasal82 This might play into it. I checked and figured out, that either the last one, or a FQDN works:

# Specifying the matching domain last works
export no_proxy=".company.com,.company.net"

# But specifying the FQDN anywhere, works too
export no_proxy=".company.net,artifactory.company.net,.company.com"

[ehuss]

I believe this may have been a regression in libcurl (see curl/curl#9884). A new version is scheduled to come out in a few weeks, so I think this will need to wait for that.

[weihanglo]

Hi, @mdegel. We've bumped curl from 7.86 to 8.0 in #11871. Could you try out the current nightly channel, or after 1.69 got released try beta channel to help us verify the behavior? Thanks!

[jschwe]

@weihanglo I'm not OP, but I also previously ran into this issue, and it seems to be fixed on nightly!

tested nightly: info: latest update on 2023-04-19, rust version 1.71.0-nightly (c609da59d 2023-04-18)

output on nightly (exit code 0):

$ cargo update
Updating `mirror` index

Output on stable:

    Updating `mirror` index
warning: spurious network error (2 tries remaining): [56] Failure when receiving data from the peer (Received HTTP code 504 from proxy after CONNECT); class=Net (12)
warning: spurious network error (1 tries remaining): [56] Failure when receiving data from the peer (Received HTTP code 504 from proxy after CONNECT); class=Net (12)
)
error: failed to get `xxx` as a dependency of package `xxx v0.1.0 (/xxxx)`

Caused by:
  failed to load source for dependency `xxx`

Caused by:
  Unable to update registry `crates-io`

Caused by:
  failed to update replaced source registry `crates-io`

Caused by:
  failed to fetch `https://xxx.yyyy.zzz.com/aaa/crates.io-index/`

Caused by:
  network failure seems to have happened
  if a proxy or similar is necessary `net.git-fetch-with-cli` may help here
  https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli

Caused by:
  [56] Failure when receiving data from the peer (Received HTTP code 504 from proxy after CONNECT); class=Net (12)

[weihanglo]

Thank you so much for the feedback! Since it is verified that the bug no longer exists, I am going to close this as resolved. Thank you all!