Python toolchain is inhermetic: Uses user site packages

[aherrmann]

Describe the bug
A rules_nixpkgs provided Python toolchain will look for Python modules in user site packages, if present, e.g. ~/.local/lib/python3.6/site-packages. This is an inhermeticity: The contents of user site packages are neither pinned by Bazel nor by Nix.

To Reproduce

• Checkout rules_nixpkgs at 6178f2a.
• Add the following to tests/BUILD.bazel

  py_binary(
    name = "foo",
   srcs = ["foo.py"]
  )
  genrule(
    name = "gen",
    outs = ["out"],
    cmd = "$(location :foo) $@",
    tools = [":foo"],
  )
  

• Add a new file tests/foo.py

  import sys
  with open(sys.argv[1], "w") as out:
      for item in sys.path:
          out.write(item + "\n");
  

• Run bazel build //tests:gen
• Inspect the output cat bazel-bin/tests/out

  /home/usr/src/rules_nixpkgs/tests
  /home/user/.cache/bazel/_bazel_aj/cf7431ac3fb5fc6f4c6a5569d7b55e4a/sandbox/linux-sandbox/2/execroot/io_tweag_rules_nixpkgs/bazel-out/host/bin/tests/foo.runfiles
  /home/user/.cache/bazel/_bazel_aj/cf7431ac3fb5fc6f4c6a5569d7b55e4a/sandbox/linux-sandbox/2/execroot/io_tweag_rules_nixpkgs/bazel-out/host/bin/tests/foo.runfiles/io_tweag_rules_nixpkgs
  /nix/store/hy65mn4wjswqih75gfr6g4q3xgqdm325-python3-3.6.6/lib/python36.zip
  /nix/store/hy65mn4wjswqih75gfr6g4q3xgqdm325-python3-3.6.6/lib/python3.6
  /nix/store/hy65mn4wjswqih75gfr6g4q3xgqdm325-python3-3.6.6/lib/python3.6/lib-dynload
  /home/user/.local/lib/python3.6/site-packages  # user site packages!
  /nix/store/hy65mn4wjswqih75gfr6g4q3xgqdm325-python3-3.6.6/lib/python3.6/site-packages
  

Expected behavior
Python should not inspect user site packages.

Environment

• OS name + version: Ubuntu 20.10
• Version of the code: Current master 6178f2a

Additional context
This is an instance of the following upstream issue: bazelbuild/bazel#4939

Thanks @guibou for reporting this.

[aherrmann]

A possible workaround is described in the upstream issue: #153.

Defining a Python wrapper in Nix that always sets PYTHONNOUSERSITE=nonempty would be another possible workaround.

[dmadisetti]

@aherrmann Did you mean to link another issue? #153 is this issue.

This is because the system binary is used over the nix one. There's an issue with bazel, genrules and PATHs: bazelbuild/bazel#12049

For the offending code, see py3wrapper.sh. The following is a work around:

genrule(
    name = "gen",
    outs =
        ["out"],
    cmd =
    ("ln -s python `dirname $(location @python39//:bin/python)`/python3;" + 
    " PATH=`dirname $(location @python39//:bin/python)`:$$PATH $(location :foo)"),
    tools =
        [
            ":foo",
            "@python39//:bin/python",
        ],
)

Edit: Nevermind, I forgot to pass in the --host_platform flag. I cannot replicate this otherwise- it might be particular to Darwin (still bazelbuild/bazel#12049)

[aherrmann]

@aherrmann Did you mean to link another issue? #153 is this issue.

Oops, yes that must have been a clipboard failure. It's been a while, so I don't remember, but I guess I was referring to this one: bazelbuild/bazel#4939 (comment)

I see that since then someone also suggested this approach: bazelbuild/bazel#4939 (comment)
Though I'm not sure if that would work on Darwin.

I cannot replicate this otherwise- it might be particular to Darwin (still bazelbuild/bazel#12049)

Hmm, at least at the time I was able to reproduce it on Linux. It's possible that the work on setting stub_shebang impacted this issue.

[AleksanderGondek]

@aherrmann I have been trying to reproduce the issue (accordingly to step provided in the description).
As long as one is using python from toolchain bootstrapped by rules_nixpkgs, the leakage does not seem to appear.

Setups on which I have tested:

• Bazel 5.1.1 from Bazelisk on Ubuntu 22.04 and RHEL 8
• Bazel 5.1.1 from nixpkgs on NixOS 21.11 and Ubuntu 22.04

Excerpt of the generated py_runtime (by rules_nixpkgs python toolchain):

py_runtime(
    name = "runtime",
    interpreter_path = "/nix/store/5bh6rpya1ar6l49vrhx1rg58dsa42906-python3-3.9.6/bin/python",
    python_version = "PY3",
    stub_shebang = "#! /nix/store/5bh6rpya1ar6l49vrhx1rg58dsa42906-python3-3.9.6/bin/python",

    visibility = ["//visibility:public"],
)

The nix-provided python interpreter seems not to be so eager to look into user site packages.

However, the issue largely remains on python provided "as-is" by Bazel by default.

Hope this helps!