Question regarding --import-mode=importlib

[Kludex]

I have this project layout:

src/
   \-> a-a
        \-> a_a
        \-> tests/
   \-> b-b
        \-> b_b
        \-> tests/

On b-b/tests I have a utility module called utils. Which means I import it with tests.utils.

As far as I understand, --import-mode=importlib doesn't add tests to the PYTHONPATH, but I have the ModuleNotFound error for tests.utils, and when I debug, what I see is that the tests imported is actually the one in a-a/tests.

So... I'm a bit confused why the first tests folder that pytest finds is added to the PYTHONPATH, or is this a Python thing?

If I add testspath, and list the tests folders it works as expected.

[nicoddemus]

Hi @Kludex

Is that reproducible in a MWE? I ask because some things are omitted from the post, for example if there are __init__.py files, or how do you execute pytest (pytest vs python -m pytest), what is the rootdir, etc.

[Kludex]

Done: Kludex/jik#1

(I think uvloop is the only issue for Windows...)

[nicoddemus]

Thanks, I can run the tests now, but I will have to continue this investigation later. Appreciate the time with the reproducer. 👍

[Kludex]

No worries. I was able to make it work with the workaround in the description anyway. I'm just really wondering why my first attempt didn't work as I'd expect.

Thanks Bruno.

[nicoddemus]

Hi @Kludex finally found the time to investigate a bit more.

Actually pytest will use resolve_pkg_root_and_module_name to derive the full test module name, so it will derive the package root based on the presence of __init__.py files on the file's directory and upwards.

In your example, for /projects/jik/src/jik-app/tests/test_app.py, it finds /projects/jik/src/jik-app/tests as the package root, and then /projects/jik/src/jik-app/tests gets added as tests to sys.modules, which results in this clashing later when importing from src/uvicorn, as tests.utils will not refer to src/uvicorn/tests/utils.py, but to src/jik-app/tests/utils.py (which does not exist) due to tests already being a module in sys.modules.

So this is indeed the cause:

Given jik-app and jik-server are separate projects, executing pytest from src might result in namespace clashes, because collecting from both at the same time will result in a single tests namespace being created.

Curiously, on main the test suite works but by accident because pythonpath is configured in pyproject.toml, which adds src/uvicorn to PYTHONPATH and causes uvicorn/tests to be import earlier... but this only masks the problem because a namespace clash might still be introduced in the future, in case someone adds some kind of testing utility module (say foo) to jik-app/tests or jik-server/tests and tries to import from there via tests.foo -- as tests in sys.modules will still be referencing uvicorn/tests.

[Kludex]

Ah, I see. Thanks Bruno. 🙏