Issue with prefixes causing a circle in version >= 20.0.0

[carsongee]

With virtualenv >= 20.0.0 I'm getting a prefixes are causing a circle RuntimeError that doesn't occur in earlier versions.

Here is the command:
python3.7 -m virtualenv --no-download --python ~/.virtualenvs/test-pytest/bin/python3.7 py37 -vvv --with-traceback
and the output:

66 setup logging to NOTSET [DEBUG report:43]
74 find interpreter for spec PythonSpec(path=/Users/carson.gee/.virtualenvs/test-pytest/bin/python3.7) [INFO builtin:43]
74 discover system for PythonInfo(spec=CPython3.7.6.final.0-64, exe=/Users/carson.gee/.virtualenvs/test-pytest/bin/python3.7, platform=darwin, version='3.7.6 (default, Dec 30 2019, 17:49:05) \n[Clang 10.0.1 (clang-1001.0.46.4)]', encoding_fs_io=utf-8-UTF-8) in /Users/carson.gee/.virtualenvs/test-pytest [DEBUG py_info:335]
76 filesystem is not case-sensitive [DEBUG info:28]
76 1: prefix=/Users/carson.gee/.virtualenvs/test-pytest, info=PythonInfo({'platform': 'darwin', 'implementation': 'CPython', 'version_info': VersionInfo(major=3, minor=7, micro=6, releaselevel='final', serial=0), 'architecture': 64, 'version': '3.7.6 (default, Dec 30 2019, 17:49:05) \n[Clang 10.0.1 (clang-1001.0.46.4)]', 'os': 'posix', 'prefix': '/Users/carson.gee/.virtualenvs/test-pytest', 'base_prefix': '/Users/carson.gee/.virtualenvs/test-pytest', 'real_prefix': '/Users/carson.gee/.virtualenvs/test-pytest', 'base_exec_prefix': '/Users/carson.gee/.virtualenvs/test-pytest', 'exec_prefix': '/Users/carson.gee/.virtualenvs/test-pytest', 'executable': '/Users/carson.gee/.virtualenvs/test-pytest/bin/python3.7', 'original_executable': '/Users/carson.gee/.virtualenvs/test-pytest/bin/python3.7', 'system_executable': None, 'has_venv': True, 'path': ['/Users/carson.gee/tmp/pytest-stereo', '/Users/carson.gee/.virtualenvs/test-pytest/lib/python37.zip', '/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7', '/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/lib-dynload', '/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/site-packages'], 'file_system_encoding': 'utf-8', 'stdout_encoding': 'UTF-8', 'sysconfig_paths': {'stdlib': '{installed_base}/lib/python{py_version_short}', 'platstdlib': '{platbase}/lib/python{py_version_short}', 'purelib': '{base}/lib/python{py_version_short}/site-packages', 'platlib': '{platbase}/lib/python{py_version_short}/site-packages', 'include': '{installed_base}/include/python{py_version_short}{abiflags}', 'scripts': '{base}/bin', 'data': '{base}'}, 'sysconfig_vars': {'abiflags': 'm', 'base': '/Users/carson.gee/.virtualenvs/test-pytest', 'platbase': '/Users/carson.gee/.virtualenvs/test-pytest', 'py_version_short': '3.7', 'installed_base': '/Users/carson.gee/.virtualenvs/test-pytest'}, 'distutils_install': {'purelib': 'lib/python3.7/site-packages', 'platlib': 'lib/python3.7/site-packages', 'headers': 'include/python3.7m/UNKNOWN', 'scripts': 'bin', 'data': ''}, 'system_stdlib': '/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7'}) [ERROR py_info:317]
77 2: prefix=/Users/carson.gee/.virtualenvs/test-pytest, info=PythonInfo({'platform': 'darwin', 'implementation': 'CPython', 'version_info': VersionInfo(major=3, minor=7, micro=6, releaselevel='final', serial=0), 'architecture': 64, 'version': '3.7.6 (default, Dec 30 2019, 17:49:05) \n[Clang 10.0.1 (clang-1001.0.46.4)]', 'os': 'posix', 'prefix': '/Users/carson.gee/.virtualenvs/test-pytest', 'base_prefix': '/Users/carson.gee/.virtualenvs/test-pytest', 'real_prefix': '/Users/carson.gee/.virtualenvs/test-pytest', 'base_exec_prefix': '/Users/carson.gee/.virtualenvs/test-pytest', 'exec_prefix': '/Users/carson.gee/.virtualenvs/test-pytest', 'executable': '/Users/carson.gee/.virtualenvs/test-pytest/bin/python3.7', 'original_executable': '/Users/carson.gee/.virtualenvs/test-pytest/bin/python3.7', 'system_executable': None, 'has_venv': True, 'path': ['/Users/carson.gee/tmp/pytest-stereo', '/Users/carson.gee/.virtualenvs/test-pytest/lib/python37.zip', '/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7', '/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/lib-dynload', '/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/site-packages'], 'file_system_encoding': 'utf-8', 'stdout_encoding': 'UTF-8', 'sysconfig_paths': {'stdlib': '{installed_base}/lib/python{py_version_short}', 'platstdlib': '{platbase}/lib/python{py_version_short}', 'purelib': '{base}/lib/python{py_version_short}/site-packages', 'platlib': '{platbase}/lib/python{py_version_short}/site-packages', 'include': '{installed_base}/include/python{py_version_short}{abiflags}', 'scripts': '{base}/bin', 'data': '{base}'}, 'sysconfig_vars': {'abiflags': 'm', 'base': '/Users/carson.gee/.virtualenvs/test-pytest', 'platbase': '/Users/carson.gee/.virtualenvs/test-pytest', 'py_version_short': '3.7', 'installed_base': '/Users/carson.gee/.virtualenvs/test-pytest'}, 'distutils_install': {'purelib': 'lib/python3.7/site-packages', 'platlib': 'lib/python3.7/site-packages', 'headers': 'include/python3.7m/UNKNOWN', 'scripts': 'bin', 'data': ''}, 'system_stdlib': '/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7'}) [ERROR py_info:318]
Traceback (most recent call last):
  File "/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/site-packages/virtualenv/__main__.py", line 47, in <module>
    run_with_catch()
  File "/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/site-packages/virtualenv/__main__.py", line 36, in run_with_catch
    run(args, options)
  File "/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/site-packages/virtualenv/__main__.py", line 19, in run
    session = cli_run(args, options)
  File "/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/site-packages/virtualenv/run/__init__.py", line 22, in cli_run
    session = session_via_cli(args, options)
  File "/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/site-packages/virtualenv/run/__init__.py", line 29, in session_via_cli
    parser = build_parser(args, options)
  File "/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/site-packages/virtualenv/run/__init__.py", line 49, in build_parser
    parser._interpreter = interpreter = discover.interpreter
  File "/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/site-packages/virtualenv/discovery/discover.py", line 44, in interpreter
    self._interpreter = self.run()
  File "/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/site-packages/virtualenv/discovery/builtin.py", line 32, in run
    return get_interpreter(self.python_spec)
  File "/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/site-packages/virtualenv/discovery/builtin.py", line 45, in get_interpreter
    for interpreter, impl_must_match in propose_interpreters(spec):
  File "/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/site-packages/virtualenv/discovery/builtin.py", line 59, in propose_interpreters
    yield PythonInfo.from_exe(spec.path), True
  File "/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/site-packages/virtualenv/discovery/py_info.py", line 293, in from_exe
    proposed = proposed._resolve_to_system(proposed)
  File "/Users/carson.gee/.virtualenvs/test-pytest/lib/python3.7/site-packages/virtualenv/discovery/py_info.py", line 319, in _resolve_to_system
    raise RuntimeError("prefixes are causing a circle {}".format("|".join(prefixes.keys())))
RuntimeError: prefixes are causing a circle /Users/carson.gee/.virtualenvs/test-pytest

I tried all versions above 20.0.0, including straight from master with the same error message.

With 16.7.9 the same command succeeds, with no issue.

Here is pip list:

pip list
Package             Version
------------------- -------------------
appdirs             1.4.3
configparser        4.0.2
contextlib2         0.6.0.post1
distlib             0.3.0.post1+dr
filelock            3.0.12
importlib-metadata  1.5.0
importlib-resources 1.0.2
packaging           20.1
pathlib2            2.3.5
pip                 10.0.2rc5+datarobot
pluggy              0.13.1
py                  1.8.1
pyparsing           2.4.6
scandir             1.10.0
setuptools          41.2.0
six                 1.14.0
toml                0.10.0
tox                 3.14.4
typing              3.7.4.1
virtualenv          16.7.9
zipp                1.1.0
You are using pip version 10.0.2rc5+datarobot, however version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

This was run on a Mac, but I've also reproduced it on CentOS 7 as well

[gaborbernat]

@carsongee if you can submit a CentOS 7 docker image replicating the issue that would be most helpful, thanks!

[carsongee]

I know this is quite convoluted, but here is a Dockerfile that will do it. The issue is that I'm working in a complex environment where sys.real_prefix is the same as the sys.prefix:

FROM python
RUN cp -Lr /usr/local /test
RUN pip install 'virtualenv<20'
RUN virtualenv test
RUN test/bin/pip install 'virtualenv==20.0.4'
RUN echo '/test' > /test/lib/python3.8/orig-prefix.txt
RUN /test/bin/python3.8 -m virtualenv --no-download --python /test/bin/python3.8 py38 -vvv

Just to add some context to the Dockerfile a bit, we aren't using an old version of virtualenv and building a new virtualenv on top of it, but we do have a site.py that sets sys.real_prefix. This is just to illustrate with a minimal set of commands

[JonnyWaffles]

Hi friends,

First time using tox and I too encountered this error on Windows with Python 3.7 and virtualenv 20.0.4

python-0.log

Edit: I tried using Tox on a completely different VM using the same source, and it worked fine. I suspect the error occurred because the prior VM has a copy of Conda installed (even though the venv I was executing from was Python 3.7). Not sure if Conda modifies install_scripts, but that's my guess.

[gaborbernat]

Hello, a fix for this issue has been released via virtualenv 20.0.6; see https://pypi.org/project/virtualenv/20.0.6/ (https://virtualenv.pypa.io/en/latest/changelog.html#v20-0-6-2020-02-26) . Please give a try and report back if your issue has not been addressed; if not, please comment here, and we'll reopen the ticket. We want to apologize for the inconvenience this has caused you and say thanks for having patience while we resolve the unexpected bugs with this new major release.