Python 3 support for external libraries

[MarkIennaco]

I have a copy of matplotlib compiled for PY3 declared as an external library for my repo.
(the url is local for testing purposes)

new_http_archive(
  name = 'python_matplotlib_repo',
  build_file = 'debian/python_matplotlib.BUILD',
  sha256 = '519bb90fe31b6de9fa3748a9f3aa3167f66c2cd6afb62c50f7b08d8e77a6cda6',
  url = 'http://miennaco-debian/matplotlib-2.0.0-1-cp34-cp34m-manylinux1_x86_64.whl',
  type = 'zip',
)

The contents of the BUILD file is:

py_library(
    name = 'matplotlib',
    visibility = ['//visibility:public'],
    srcs = glob(['matplotlib/**/*.py']),
    data = [
        ':matplotlib_data',
    ],
    deps = [
        '@python_cycler_repo//:cycler',
        '@python_pyparsing_repo//:pyparsing',
    ],
    srcs_version = 'PY3',
)

filegroup(
    name = 'matplotlib_data',
    srcs = glob([
        'matplotlib/mpl-data/**/*',
        'matplotlib/**/*.so',
        'matplotlib/.libs/*',
    ]),
)

When I try to build a target that depends on the library:

 bazel test vehicle/logging/analysis:timing_report_plotter_testERROR: /home/miennaco/.cache/bazel/_bazel_miennaco/af311f486463d9a61a3fdd9b3eb0bcd4/external/python_matplotlib_repo/BUILD.bazel:1:1: in py_library rule @python_matplotlib_repo//:matplotlib: Rule '@python_matplotlib_repo//:matplotlib' need to be converted to Python 2 (not yet implemented).
ERROR: Analysis of target '//vehicle/logging/analysis:timing_report_plotter_test' failed; build aborted.
INFO: Elapsed time: 0.450s
ERROR: Couldn't start the build. Unable to run tests.

The target in question has:

  srcs_version = 'PY3',
  default_python_version = 'PY3',

But bazel insists on building the library with python 2.

Adding --force_python PY3 to the build allows this target to built (but breaks all the python 2 projects in the repository)

Environment info

• Operating System:
  $ uname --all
  Linux miennaco-debian 3.16.0-4-amd64 Update the Readme to give a short description of bazel. #1 SMP Debian 3.16.39-1+deb8u1 (2017-02-22) x86_64 GNU/Linux

• Bazel version (output of bazel info release):
  Local build based on bazel-0.5.0-rc2

[brandjon]

That error message occurs when you attempt to build a PY3(only) target in PY2 mode. It sounds like the library got built from within a py_binary that either had default_python_version = "PY2" or else didn't set default_python_version (in which case it... defaults... to "PY2").

Can you provide more info about how your timing_report_plotter_test target depends on the matplotlib target, and where the default_python_version is in the chain? Does matplotlib build if you specify it directly on the command line along with --force_python=PY3, i.e. bazel build //some_path:matplotlib --force_python=PY3?

See also #6444 for an index of Python 2/3 issues.

[brandjon]

The messiness of having PY2 and PY3 targets in the same build should be addressed by #6583. You may also be interested in the move to make PY3 default, which is #6647.