Browsing for Python interpreter follows symlinks when it shouldn't

[gjoseph92]

Environment data

• VS Code version: 1.58.2
• Extension version (available under the Extensions sidebar): v2021.6.944021595
• OS and version: macOS 11.4
• Python version (& distribution if applicable, e.g. Anaconda): 3.9.1, pyenv (source)
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): python -m venv
• Relevant/affected Python packages and their versions: XXX
• Relevant/affected Python-related VS Code extensions and their versions: XXX
• Value of the python.languageServer setting: Pylance

[NOTE: If you suspect that your issue is related to the Microsoft Python Language Server (python.languageServer: 'Microsoft'), please download our new language server Pylance from the VS Code marketplace to see if that fixes your issue]

Expected behaviour

When selecting the path to a Python interpreter through the native file browser, symlinks are treated as normal files (you can select a symlink as the interpreter).

Actual behaviour

When selecting the path to a Python interpreter through the native file browser, symlinks are resolved. If venv/bin/python is a symlink to /usr/bin/python, VS Code would pick /usr/bin/python as the interpreter, not venv/bin/python as intended.

Steps to reproduce:

mkdir test-env && cd test-env
python -m venv env
code .

Now select the Python interpreter in VS Code. test-env/env/bin/python will not show up—this is already noted in #16409 (also somewhat #16704, #16613). So to select my environment manually, I try "Enter interpreter path...", then "Find...".

Using the native macOS file picker, I select ./env/bin/python (which is a symlink to ~/.pyenv/versions/3.9.1/bin/python). But VS Code (or the native file picker?) seems to follow this symlink and select ~/.pyenv/versions/3.9.1/bin/python as the path to the interpreter.

Note that entering ./env/bin/python as the interpreter path (rather than using the native file picker) works as expected. If the native file picker always resolves symlinks, maybe that option should be removed?

Also note that creating the virtual environment with python -m venv --copies env makes selecting the interpreter through the native file picker work as expected.

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

User belongs to experiment group 'pythonaacf'
User belongs to experiment group 'pythonJoinMailingListVar3'
User belongs to experiment group 'pythonJediLSP'
User belongs to experiment group 'pythonDiscoveryModuleWithoutWatcher'
User belongs to experiment group 'pythonTensorboardExperiment'
User belongs to experiment group 'pythonSurveyNotification'
User belongs to experiment group 'PythonPyTorchProfiler'
User belongs to experiment group 'pythonDeprecatePythonPath'
Python interpreter path: /usr/bin/python3
Starting Pylance language server.
Error 2021-07-16 15:24:52: Failed to check if file needs to be fixed [EntryNotFound (FileSystemError): Unable to read file '/Users/gabe/dev/test-python-venvs/test-env/.vscode/settings.json' (Error: Unable to resolve non-existing file '/Users/gabe/dev/test-python-venvs/test-env/.vscode/settings.json')
	at _handleError (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/services/extensions/node/extensionHostProcess.js:84:167834)
	at runMicrotasks (<anonymous>)
	at processTicksAndRejections (internal/process/task_queues.js:93:5)
	at async y.readText (/Users/gabe/.vscode/extensions/ms-python.python-2021.6.944021595/out/client/extension.js:9:343940)
	at async p.doesFileNeedToBeFixed (/Users/gabe/.vscode/extensions/ms-python.python-2021.6.944021595/out/client/extension.js:59:581700)
	at async /Users/gabe/.vscode/extensions/ms-python.python-2021.6.944021595/out/client/extension.js:59:580827
	at async Promise.all (index 1)
	at async p.getFilesToBeFixed (/Users/gabe/.vscode/extensions/ms-python.python-2021.6.944021595/out/client/extension.js:59:580773)
	at async p.updateTestSettings (/Users/gabe/.vscode/extensions/ms-python.python-2021.6.944021595/out/client/extension.js:59:580400)] {
  code: 'FileNotFound'
}
Python interpreter path: ~/.pyenv/versions/3.9.1/bin/python3.9

[kimadeline]

Hi @gjoseph92, thank you for reaching out!

There are a couple of layers in your issue here, let me walk through them:

Now select the Python interpreter in VS Code. test-env/env/bin/python will not show up

This is expected: Interpreter selection will happen on activation, but the extension won't activate when you open a folder, you need to open a Python file for that.

Secondly, we are also gradually rolling out an updated way of auto-selecting interpreters and sorting them in the "Select Interpreter" dropdown (#16520), starting with today's release of the Python extension (2021.7.1050252941). You can opt into this experiment by adding the following to your settings.json:

"python.experiments.optInto": [
        "pythonSortEnvs"
],

If you are in this experiment and you open a Python file, the virtual environment located in your workspace should be selected. Your virtual environment should also appear at the top of the list of environments as well, since we now sort workspace environments to appear higher.

If the native file picker always resolves symlinks, maybe that option should be removed?

This was reported earlier in #13603. Currently there's no way for extensions to disable that option since VS Code doesn't expose it. Feel free to upvote the upstream issue to express your interest in this issue!

Closing as a duplicate of #13603.

[gjoseph92]

Thanks @kimadeline for the follow-up!