Providing basic advice on network related failures

[pradyunsg]

Is your feature request related to a problem? Please describe.

Currently, many users of pip see weird error messages regarding network connectivity. While these messages are helpful for debugging the situation, they aren't exactly good UX.

Describe the solution you'd like

It might be nice if pip tried to cover some bases and suggested the user to look into common mistakes like not using a proxy when it's needed. Just an idea for discussion.

Additional context

@pypa/pip-committers Actually, I was wondering...

There's quite a few situations where we're presenting a Traceback today and we could improve the user experience by printing a message and the error and printing the Traceback only when verbose, like we did in #5239.

[pradyunsg]

cmdargs: ['/Users/pradyunsg/Projects/pip/.tox/py36/bin/python', '/Users/pradyunsg/Projects/pip/tools/tox_pip.py', 'install', '-r/Users/pradyunsg/Projects/pip/tools/tests-requirements.txt']

Collecting freezegun (from -r /Users/pradyunsg/Projects/pip/tools/tests-requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/1b/78/feef0b235f1fed24aa5e617dee51f16d7cfd236bdacd0319718ce4706092/freezegun-0.3.10-py2.py3-none-any.whl
Collecting mock (from -r /Users/pradyunsg/Projects/pip/tools/tests-requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting pretend (from -r /Users/pradyunsg/Projects/pip/tools/tests-requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/49/1f/3d4f0579913edd3ad5b23ad52fcc42531cb736ad52af2ba6c057da8785b6/pretend-1.0.9-py2.py3-none-any.whl
Collecting pytest (from -r /Users/pradyunsg/Projects/pip/tools/tests-requirements.txt (line 4))
  Using cached https://files.pythonhosted.org/packages/dd/e5/1ce7de3e87ec499a056800fa0d7a689d6502d791c44eb1315a6ecadcb333/pytest-3.8.0-py2.py3-none-any.whl
Collecting pytest-cov (from -r /Users/pradyunsg/Projects/pip/tools/tests-requirements.txt (line 5))
  Using cached https://files.pythonhosted.org/packages/30/0a/1b009b525526cd3cd9f52f52391b426c5a3597447be811a10bcb1f6b05eb/pytest_cov-2.6.0-py2.py3-none-any.whl
Collecting pytest-rerunfailures (from -r /Users/pradyunsg/Projects/pip/tools/tests-requirements.txt (line 6))
  Using cached https://files.pythonhosted.org/packages/c5/81/2dc013556d53c9f9a1955e66f8620a05690eb6de9efc9fdb6cd748c66d24/pytest_rerunfailures-4.1-py2.py3-none-any.whl
Collecting pytest-timeout (from -r /Users/pradyunsg/Projects/pip/tools/tests-requirements.txt (line 7))
  Using cached https://files.pythonhosted.org/packages/ae/77/3b714fcfda89925be29f5cdea5b6199912265f54dc23b9af7d8c588e1830/pytest_timeout-1.3.2-py2.py3-none-any.whl
Collecting pytest-xdist (from -r /Users/pradyunsg/Projects/pip/tools/tests-requirements.txt (line 8))
  Using cached https://files.pythonhosted.org/packages/06/9f/d742d21f278a40c146362e69bc3d92d80b713368a4d0c48565b5c7820611/pytest_xdist-1.23.0-py2.py3-none-any.whl
Collecting pyyaml (from -r /Users/pradyunsg/Projects/pip/tools/tests-requirements.txt (line 9))
Collecting scripttest (from -r /Users/pradyunsg/Projects/pip/tools/tests-requirements.txt (line 10))
Collecting virtualenv from https://github.com/pypa/virtualenv/archive/master.zip#egg=virtualenv (from -r /Users/pradyunsg/Projects/pip/tools/tests-requirements.txt (line 11))
  Downloading https://github.com/pypa/virtualenv/archive/master.zip
Exception:
Traceback (most recent call last):
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 331, in _error_catcher
    yield
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 640, in read_chunked
    chunk = self._handle_chunk(amt)
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 586, in _handle_chunk
    value = self._fp._safe_read(amt)
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 70, in _safe_read
    data = self.__fp._safe_read(amt)
  File "/Users/pradyunsg/.pyenv/versions/3.6.5/lib/python3.6/http/client.py", line 612, in _safe_read
    chunk = self.fp.read(min(amt, MAXAMOUNT))
  File "/Users/pradyunsg/.pyenv/versions/3.6.5/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
  File "/Users/pradyunsg/.pyenv/versions/3.6.5/lib/python3.6/ssl.py", line 1009, in recv_into
    return self.read(nbytes, buffer)
  File "/Users/pradyunsg/.pyenv/versions/3.6.5/lib/python3.6/ssl.py", line 871, in read
    return self._sslobj.read(len, buffer)
  File "/Users/pradyunsg/.pyenv/versions/3.6.5/lib/python3.6/ssl.py", line 631, in read
    v = self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_internal/basecommand.py", line 141, in main
    status = self.run(options, args)
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 299, in run
    resolver.resolve(requirement_set)
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_internal/resolve.py", line 102, in resolve
    self._resolve_one(requirement_set, req)
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_internal/resolve.py", line 256, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_internal/resolve.py", line 209, in _get_abstract_dist_for
    self.require_hashes
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 283, in prepare_linked_requirement
    progress_bar=self.progress_bar
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_internal/download.py", line 836, in unpack_url
    progress_bar=progress_bar
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_internal/download.py", line 673, in unpack_http_url
    progress_bar)
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_internal/download.py", line 897, in _download_http_url
    _download_url(resp, link, content_file, hashes, progress_bar)
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_internal/download.py", line 619, in _download_url
    consume(downloaded_chunks)
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_internal/utils/misc.py", line 844, in consume
    deque(iterator, maxlen=0)
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_internal/download.py", line 585, in written_chunks
    for chunk in chunks:
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_internal/utils/ui.py", line 159, in iter
    for x in it:
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_internal/download.py", line 574, in resp_read
    decode_content=False):
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 461, in stream
    for line in self.read_chunked(amt, decode_content=decode_content):
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 665, in read_chunked
    self._original_response.close()
  File "/Users/pradyunsg/.pyenv/versions/3.6.5/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/pradyunsg/Projects/pip/.tox/py36/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 336, in _error_catcher
    raise ReadTimeoutError(self._pool, None, 'Read timed out.')
pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='codeload.github.com', port=443): Read timed out.

Instead of the entire error traceback, just being able to show the relevant information would be useful.

[chrahunt]

Another example from #4723:

NewConnectionError('<pip._vendor.requests.packages.urllib3.conne
ction.VerifiedHTTPSConnection object at 0x00000000035E1B70>: Failed to establish
a new connection: [Errno 11004] getaddrinfo failed',)': /simple/robotframework/

As mentioned in #1876, these kinds of messages could be better, e.g.

Failed to get address for <hostname>! Check your network connectivity and DNS settings. Details: [Errno 11004] getaddrinfo failed',)': /simple/robotframework/

[gutsytechster]

Where could these errors be incorporated? On looking through the codebase, I think, this would be the place to do so- https://github.com/pypa/pip/blob/master/src/pip/_internal/network/download.py#L190?

[uranusjr]

There are two places that can raise exceptions. The place you mentioned would raise if the connection can’t be established (e.g. no Internet whatsoever). The other is pip._internal.network.utils.response_chunks(), where the response content is read out.

I think the best way to handle these is to introduce a new exception subclass, and raise it in these locations. For the former, I’d actually prefer we replace the raise_for_status() in _http_get_download() directly and avoid raising HTTPError altogether (for another reason #7486—this; in fact all raise_for_status() occurences could be replaced by this error message). The latter would involve catching errors from urllib3 and raise the exception subclass.

The new exception subclass should be handled in BaseCommand._main() along with InstallationError etc.

[gutsytechster]

Thank you for the response @uranusjr. Could you clear these doubts?

• When you say about the former, do you actually mean the error which would be raised when a connection can't be established? And for that, you prefer to replace all occurrences of raise_for_status() with the similar message mentioned here Providing basic advice on network related failures #5380 (comment). However, we would still require to raise the new exception subclass with that message, won't we?

• And when you talk about the latter, do you meant the pip._internal.network.utils.response_chunks() piece of code, where the exceptions would be caught based on the urllib3 exceptions?

And just to know, why do we use urllib3 and requests separately. I mean, won't anyone of them would be sufficient to do the tasks of other one?

[uranusjr]

Yes to both your questions. I would be very happy if we replace all raise_for_status() occurrences, but this doesn’t need to be done all at once.

pip does not call urllib3 directly, only requests. But requests uses urllib3 internally, and sometimes just lets urllib3 errors bubble up to the call site without catching/re-raising them. I don’t want to deal with this either, but it’s just the world we live in :(

[gutsytechster]

@uranusjr when you say to replace all occurrences of raise_for_status. AFAICT, the main aim here is not to raise HTTPError, rather a subclass of our own exception. But we still would need to evaluate the response just as the raise_for_status does, won't we? And for that, there would be a similar method implementation just as raise_for_status with the only difference in the final exception raised?

[uranusjr]

Yup.

[zahlman]

A little disappointing to see this issue go nowhere for 6 years, because I have seen that users are quite seriously put off by this.

Here's a clear (anonymized) example of a poor message that I constantly see people report on Stack Overflow, on the Discourse and elsewhere:

WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. (read timeout=15)")': /path/to/some-package-1.0.0-py3-none-any.whl.metadata

Almost all of this information is useless to the user in context:

• the user knows which package Pip is trying to install
• I've literally never seen connect, read, redirect or status have any values other than None when an error like this is reported, and it wouldn't help understand what to do about the problem anyway
• The classes Retry, ReadTimeoutError and HTTPSConnectionPool are implementation details in code that the user didn't write and isn't trying to call upon (only use, as an ordinary software user)
• Some version of "read timeout" appears three times, and "retry" appears twice
• The port number information isn't important unless there's an actual problem to solve (as opposed to an intermittent connection glitch)

The useful information is hard to extract and strangely labelled from the user's perspective:

• The 4 for total=4 tells us how many further times Pip will re-try the connection
• The server didn't respond to Pip, which is described as a "read timeout" because Pip set a time limit to read data from the server - that's standard terminology for people accustomed to writing networking code, but otherwise obscure
• Pip spent 15 seconds waiting for the connection; that's the timeout value, and units are not mentioned
• It's a metadata file, as indicated by the extension

And it's especially unpleasant to see all this text repeated five times (because retrying often doesn't solve the problem), culminating in this error (which essentially repeats it a sixth time):

ERROR: Could not install packages due to an OSError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries exceeded with url: /path/to/some-package-1.0.0-py3-none-any.whl.metadata (Caused by ReadTimeoutError("HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. (read timeout=15)"))

And none of this concretely suggests what might be done about it.

Here's a rough cut at how it should look instead:

WARNING: having difficulty connecting to files.pythonhosted.org via HTTPS (will try 4 more times). The server didn't respond within 15 seconds.
WARNING: having difficulty connecting to files.pythonhosted.org via HTTPS (will try 3 more times). The server didn't respond within 15 seconds.
WARNING: having difficulty connecting to files.pythonhosted.org via HTTPS (will try 2 more times). The server didn't respond within 15 seconds.
WARNING: having difficulty connecting to files.pythonhosted.org via HTTPS (will try 1 more time). The server didn't respond within 15 seconds.
WARNING: having difficulty connecting to files.pythonhosted.org via HTTPS (giving up). The server didn't respond within 15 seconds.
ERROR: Failed to install some-package==1.0.0: couldn't retrieve metadata because of a connection issue.
Please make sure that you are connected to the Internet and permitted to make outgoing connections to files.pythonhosted.org on port 443 (check your firewall and/or proxy settings) before trying again.

[pfmoore]

A little disappointing to see this issue go nowhere for 6 years, because I have seen that users are quite seriously put off by this.

The main reason is that no-one has offered a PR improving this. Pip is a volunteer driven project with limited maintainer resource, so we rely heavily on community contributions.

[zahlman]

Ah, I should realize that much.

Do the suggested messages look right, at least? It would be rather unpleasant to start searching around the codebase for how to change this, and then start working on a feature branch, only to find out that the change isn't actually wanted.

[pfmoore]

The proposed output certainly seems good.

[pradyunsg]

x-ref urllib3/urllib3#2580 -- these messages don't originate out of pip.

I don't have bandwidth to argue against "go nowhere for 6 years", so I'll say that it's a bit disappointing for me too that this hasn't improved. :)