Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NO_BUILD] Added first draft of python wheel documentation #4325

Merged
merged 21 commits into from
Jul 30, 2021

Conversation

corkyw10
Copy link
Contributor

@corkyw10 corkyw10 commented Jun 21, 2021

Python Wheel Documentation

I have written the first draft of the Python Wheel documentation. I have included sections that tell the user how .egg files were used prior to versions 0.9.12 and to select the right documentation for the version being used if necessary.

Since the wheel is only supported in Ubuntu 18.04+ I have removed the sections that refer to separate packages that need to be installed for previous Ubuntu versions in the Linux build docs. Please double check this section in case the previous information is still needed.

Auditwheel may not be required as a dependency, this can be removed if needed.


This change is Reviewable

@corkyw10 corkyw10 requested a review from joel-mb June 21, 2021 11:05
@corkyw10 corkyw10 self-assigned this Jun 21, 2021
@corkyw10 corkyw10 added the documentation Requires changes/additions to documentation label Jun 21, 2021
@corkyw10 corkyw10 added this to the 0.9.12 milestone Jun 21, 2021
@corkyw10 corkyw10 changed the title Added first draft of python wheel documentation [NO_BUILD] Added first draft of python wheel documentation Jun 21, 2021
Copy link
Contributor

@joel-mb joel-mb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 6 files at r1.
Reviewable status: 1 of 6 files reviewed, 3 unresolved discussions (waiting on @corkyw10)


Docs/build_linux.md, line 26 at r1 (raw file):

* __Ubuntu 18.04+.__ CARLA versions 0.9.12+ require a Linux system that uses a version of __glibc >= 2.27__.

With the last changes, I think we should not modify this section. The transition from eggs to wheels doesn't affect this.

When building from source, the previous requisites are still valid.
The glibc>=2.27 requirement applies when installing the distributed Python client. Our Jenkins machine is using Ubuntu 18.04 and we will distribute the CARLA wheels targeting the manylinux_2_27_x86_64 platform tag. Therefore, pip install carla will only work in Linux distros withgibc>=2.27. When building from source everything should remain the same.


Docs/build_linux.md, line 208 at r1 (raw file):

Previous versions of CARLA created a `.egg` file containing the CARLA client library. __In versions 0.9.12+ this behavior changes significantly; `.egg` files are no longer used.__ `make PythonAPI` will install the library using `pip` and a `.whl` file. Every time you run `make PythonAPI` the previous library you had installed will be uninstalled and a new one will be installed according to the source code you are using.

Now it is possible to change this behaviour. I've added a couple of arguments to the make PythonAPI command:

  • --no-install-wheel. If specified, doesn't install the wheel. By default this is False
  • --target-wheel-platform. If specified, we try to repair the wheel with auditwheel targeting this platform.

Docs/build_system.md, line 81 at r1 (raw file):

Compiled using Python's `setuptools` ("setup.py"). Currently requires the following to be installed in the machine: Python, libpython-dev, and
libboost-python-dev, pip>=20.3, wheel and auditwheel.

I think we can remove libboost-python-dev from this list as we are building it from source. @bernatx can you confirm that?

Copy link
Contributor Author

@corkyw10 corkyw10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 1 of 6 files reviewed, 3 unresolved discussions (waiting on @corkyw10 and @joel-mb)


Docs/build_linux.md, line 26 at r1 (raw file):

Previously, joel-mb wrote…
* __Ubuntu 18.04+.__ CARLA versions 0.9.12+ require a Linux system that uses a version of __glibc >= 2.27__.

With the last changes, I think we should not modify this section. The transition from eggs to wheels doesn't affect this.

When building from source, the previous requisites are still valid.
The glibc>=2.27 requirement applies when installing the distributed Python client. Our Jenkins machine is using Ubuntu 18.04 and we will distribute the CARLA wheels targeting the manylinux_2_27_x86_64 platform tag. Therefore, pip install carla will only work in Linux distros withgibc>=2.27. When building from source everything should remain the same.

Done.


Docs/build_linux.md, line 208 at r1 (raw file):

Previously, joel-mb wrote…
Previous versions of CARLA created a `.egg` file containing the CARLA client library. __In versions 0.9.12+ this behavior changes significantly; `.egg` files are no longer used.__ `make PythonAPI` will install the library using `pip` and a `.whl` file. Every time you run `make PythonAPI` the previous library you had installed will be uninstalled and a new one will be installed according to the source code you are using.

Now it is possible to change this behaviour. I've added a couple of arguments to the make PythonAPI command:

  • --no-install-wheel. If specified, doesn't install the wheel. By default this is False
  • --target-wheel-platform. If specified, we try to repair the wheel with auditwheel targeting this platform.

Done.

Copy link
Contributor

@joel-mb joel-mb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 1 of 6 files reviewed, 4 unresolved discussions (waiting on @corkyw10)


Docs/build_linux.md, line 68 at r3 (raw file):

CARLA requires some Python dependencies to be installed with `pip version >= 20.3`.

The dependencies are not required to be installed with pip>=20.3. pip>=20.3 is necessary to install the CARLA PythonAPI with pip. I think it is fine to say here to update pip to 20.3 or greater. But maybe omit the part CARLA requires some Python dependencies to be installed with pip version >= 20.3. Also it would be necessary to update pip3


Docs/build_linux.md, line 223 at r3 (raw file):

Every time you run `make PythonAPI` a `.whl` file containing the CARLA client library will be built and installed for your system. This `.whl` cannot be distributed as it is specific for your OS. If you had a previous library installed, it will be uninstalled, and a new one will be installed according to the source code you are using. You can modify this behavior with the following flags:

This has been changed now. make PythonAPI will build both the egg and the wheel. The wheel is not installed automatically. Also, the --no-install-wheel command is not longer available.


Docs/build_windows.md, line 50 at r3 (raw file):

CARLA requires some Python dependencies to be installed with `pip version >= 20.3`.

Same comment as in Linux

Copy link
Contributor Author

@corkyw10 corkyw10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 6 files reviewed, 4 unresolved discussions (waiting on @corkyw10 and @joel-mb)


Docs/build_linux.md, line 68 at r3 (raw file):

Previously, joel-mb wrote…
CARLA requires some Python dependencies to be installed with `pip version >= 20.3`.

The dependencies are not required to be installed with pip>=20.3. pip>=20.3 is necessary to install the CARLA PythonAPI with pip. I think it is fine to say here to update pip to 20.3 or greater. But maybe omit the part CARLA requires some Python dependencies to be installed with pip version >= 20.3. Also it would be necessary to update pip3

Done.


Docs/build_linux.md, line 223 at r3 (raw file):

Previously, joel-mb wrote…
Every time you run `make PythonAPI` a `.whl` file containing the CARLA client library will be built and installed for your system. This `.whl` cannot be distributed as it is specific for your OS. If you had a previous library installed, it will be uninstalled, and a new one will be installed according to the source code you are using. You can modify this behavior with the following flags:

This has been changed now. make PythonAPI will build both the egg and the wheel. The wheel is not installed automatically. Also, the --no-install-wheel command is not longer available.

Done.

Copy link
Contributor

@joel-mb joel-mb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 6 files reviewed, 12 unresolved discussions (waiting on @corkyw10 and @joel-mb)


Docs/build_faq.md, line 414 at r5 (raw file):

```sh
# If you installed the .whl
# Python 3
pip3 uninstall <wheel-file-name>.whl
# Python 2
pip uninstall <wheel-file-name>.whl
# If you installed the PyPi package
# Python 3
pip3 uninstall carla
# Python 2
pip uninstall carla

To uninstall the package is always the same:

pip/pip3 uninstall carla

It doesn't matter if you install it using directly the wheel or with pypi


Docs/build_linux.md, line 223 at r5 (raw file):

Python versions 2.7, 3.6, 3.7, and 3.8 are supported

I wouldn't say that only this python versions are supported. When compiling from source you can compile against any version of python. It is true that we will only distribute the PythonAPI for those python versions but I would omit this information here.


Docs/build_linux.md, line 234 at r5 (raw file):

The `.egg` file should be added to your `PYTHONPATH`. All of CARLA's example scripts automatically [look for this file](build_system.md#versions-prior-to-0912) when importing CARLA.

I would rewrite this sentence. When running the CARLA's example scripts it is not necessary to add anything to the PYTHONPATH


Docs/build_system.md, line 90 at r5 (raw file):

Python 2.7, 3.6, 3.7, and 3.8 are supported.

Same as before. When compiling from source you can target other python versions.


Docs/build_windows.md, line 206 at r5 (raw file):

Python versions 2.7, 3.6, 3.7, and 3.8 are supported.

Same as in Linux


Docs/build_windows.md, line 217 at r5 (raw file):

he `.egg` file will automatically be added to your `PYTHONPATH`. All of CARLA's example scripts automatically [look for this file](build_system.md#versions-prior-to-0912) when importing CARLA.

Same as in Linux.


Docs/start_quickstart.md, line 22 at r5 (raw file):

If running on Linux, the OS must support __glibc >= 2.27__ (Ubuntu 18.04+)

This is only for the client when installing it from the PyPI index. Not sure if we should add this requirement here.


Docs/start_quickstart.md, line 43 at r5 (raw file):

CARLA requires some dependencies for the client library. Install the dependencies according to your operating system:

It is not necessary to install wheel or auditwheel to install the CARLA PythonAPI via pip. These packages are only needed when building the wheel.


Docs/start_quickstart.md, line 160 at r5 (raw file):

Python versions 2.7, 3.6, 3.7, and 3.8

I would omit the python versions here. In CARLA packages, we will provide the same python versions as for the egg files.


Docs/start_quickstart.md, line 271 at r5 (raw file):

```sh
# If you installed the .whl
# Python 3
pip3 uninstall <wheel-file-name>.whl
# Python 2
pip uninstall <wheel-file-name>.whl
# If you installed the PyPi package
# Python 3
pip3 uninstall carla
# Python 2
pip uninstall carla

Same as above. You can uninstall the package using pip/pip3 uninstall carla. It doesn't matter if you used the wheel or PyPI index to install it.


Docs/tuto_G_retrieve_data.md, line 844 at r5 (raw file):

 Commented import areas only required if running versions of CARLA prior to 0.9.12

I think we can leave this as before as the egg will be available for 0.9.12+

Copy link
Contributor Author

@corkyw10 corkyw10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 6 files reviewed, 12 unresolved discussions (waiting on @corkyw10 and @joel-mb)


Docs/build_faq.md, line 414 at r5 (raw file):

Previously, joel-mb wrote…
```sh
# If you installed the .whl
# Python 3
pip3 uninstall <wheel-file-name>.whl
# Python 2
pip uninstall <wheel-file-name>.whl
# If you installed the PyPi package
# Python 3
pip3 uninstall carla
# Python 2
pip uninstall carla

To uninstall the package is always the same:

pip/pip3 uninstall carla

It doesn't matter if you install it using directly the wheel or with pypi

Done.


Docs/build_linux.md, line 223 at r5 (raw file):

Previously, joel-mb wrote…
Python versions 2.7, 3.6, 3.7, and 3.8 are supported

I wouldn't say that only this python versions are supported. When compiling from source you can compile against any version of python. It is true that we will only distribute the PythonAPI for those python versions but I would omit this information here.

Done.


Docs/build_linux.md, line 234 at r5 (raw file):

Previously, joel-mb wrote…
The `.egg` file should be added to your `PYTHONPATH`. All of CARLA's example scripts automatically [look for this file](build_system.md#versions-prior-to-0912) when importing CARLA.

I would rewrite this sentence. When running the CARLA's example scripts it is not necessary to add anything to the PYTHONPATH

Done.


Docs/build_system.md, line 90 at r5 (raw file):

Previously, joel-mb wrote…
Python 2.7, 3.6, 3.7, and 3.8 are supported.

Same as before. When compiling from source you can target other python versions.

Done.


Docs/build_windows.md, line 206 at r5 (raw file):

Previously, joel-mb wrote…
Python versions 2.7, 3.6, 3.7, and 3.8 are supported.

Same as in Linux

Done.


Docs/build_windows.md, line 217 at r5 (raw file):

Previously, joel-mb wrote…
he `.egg` file will automatically be added to your `PYTHONPATH`. All of CARLA's example scripts automatically [look for this file](build_system.md#versions-prior-to-0912) when importing CARLA.

Same as in Linux.

Done.


Docs/start_quickstart.md, line 22 at r5 (raw file):

Previously, joel-mb wrote…
If running on Linux, the OS must support __glibc >= 2.27__ (Ubuntu 18.04+)

This is only for the client when installing it from the PyPI index. Not sure if we should add this requirement here.

Done.


Docs/start_quickstart.md, line 43 at r5 (raw file):

Previously, joel-mb wrote…
CARLA requires some dependencies for the client library. Install the dependencies according to your operating system:

It is not necessary to install wheel or auditwheel to install the CARLA PythonAPI via pip. These packages are only needed when building the wheel.

Done.


Docs/start_quickstart.md, line 160 at r5 (raw file):

Previously, joel-mb wrote…
Python versions 2.7, 3.6, 3.7, and 3.8

I would omit the python versions here. In CARLA packages, we will provide the same python versions as for the egg files.

Done.


Docs/start_quickstart.md, line 271 at r5 (raw file):

Previously, joel-mb wrote…
```sh
# If you installed the .whl
# Python 3
pip3 uninstall <wheel-file-name>.whl
# Python 2
pip uninstall <wheel-file-name>.whl
# If you installed the PyPi package
# Python 3
pip3 uninstall carla
# Python 2
pip uninstall carla

Same as above. You can uninstall the package using pip/pip3 uninstall carla. It doesn't matter if you used the wheel or PyPI index to install it.

Done.


Docs/tuto_G_retrieve_data.md, line 844 at r5 (raw file):

Previously, joel-mb wrote…
 Commented import areas only required if running versions of CARLA prior to 0.9.12

I think we can leave this as before as the egg will be available for 0.9.12+

Done.

Copy link
Contributor

@joel-mb joel-mb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 6 files reviewed, 3 unresolved discussions (waiting on @corkyw10 and @joel-mb)


Docs/build_faq.md, line 147 at r7 (raw file):

CARLA used `.egg` files for the client library in versions prior to 0.9.12. If you are using 0.9.12+, `.whl` files are used instead and the information in this section will not be relevant to you.

In 0.9.12+ we also have eggs. I would copy and paste the sentence you have in the Windows section:

If you are using 0.9.12+, there are several methods to use/install the client library. If you are using one of the newer methods for the client library (.whl or PyPi download) the information in this section will not be relevant to you.

Docs/build_windows.md, line 211 at r7 (raw file):

Optionally, to compile the PythonAPI for a specific version of Python, run the below command in the root CARLA directory.
```sh
    # Delete versions as required
    make PythonAPI ARGS="--python-version=2.7, 3.6, 3.7, 3.8"

I just remember this option is not available in Windows. The PythonAPI is built against the default python3 installation. We can just remove this part of the section.


Docs/tuto_G_retrieve_data.md, line 844 at r5 (raw file):

Previously, corkyw10 (Corinne) wrote…

Done.

Please, remove also the # in the code.

Copy link
Contributor Author

@corkyw10 corkyw10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 6 files reviewed, 3 unresolved discussions (waiting on @corkyw10 and @joel-mb)


Docs/build_faq.md, line 147 at r7 (raw file):

Previously, joel-mb wrote…
CARLA used `.egg` files for the client library in versions prior to 0.9.12. If you are using 0.9.12+, `.whl` files are used instead and the information in this section will not be relevant to you.

In 0.9.12+ we also have eggs. I would copy and paste the sentence you have in the Windows section:

If you are using 0.9.12+, there are several methods to use/install the client library. If you are using one of the newer methods for the client library (.whl or PyPi download) the information in this section will not be relevant to you.

Done.


Docs/build_windows.md, line 211 at r7 (raw file):

Previously, joel-mb wrote…
Optionally, to compile the PythonAPI for a specific version of Python, run the below command in the root CARLA directory.
```sh
    # Delete versions as required
    make PythonAPI ARGS="--python-version=2.7, 3.6, 3.7, 3.8"

I just remember this option is not available in Windows. The PythonAPI is built against the default python3 installation. We can just remove this part of the section.

Done.


Docs/tuto_G_retrieve_data.md, line 844 at r5 (raw file):

Previously, joel-mb wrote…

Please, remove also the # in the code.

Done.

@corkyw10 corkyw10 marked this pull request as ready for review July 29, 2021 06:43
@corkyw10 corkyw10 requested a review from a team as a code owner July 29, 2021 06:43
Copy link
Contributor

@joel-mb joel-mb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 6 files reviewed, 2 unresolved discussions (waiting on @corkyw10 and @joel-mb)


Docs/build_linux.md, line 250 at r9 (raw file):

>This `.whl` file cannot be distributed as it is built specifically for your OS. To build a `.whl` for distribution, you can target a different system by using the following flag:
>- `make PythonAPI --target-wheel-platform="<platform>"` will build and repair the `.whl` file using `auditwheel` for a specific [platform](https://github.com/pypa/manylinux). External shared dependencies will be included in the file. The packaged version of the CARLA `.whl` targets the `manylinux_2_27_x86_64` platform.

Please, remove this part.


Docs/build_windows.md, line 50 at r9 (raw file):

Starting with CARLA 0.9.12, users have the option to install the CARLA Python API using `pip` or `pip3`. Version 20.3 or higher is required. To check if you have a suitable version, run the following command:
```sh
# For Python 3
pip3 -V
# For Python 2
pip -V

If you need to upgrade:

# For Python 3
pip3 install --upgrade pip
# For Python 2
pip install --upgrade pip

You must install the following Python dependencies:

# Python 3
pip3 install --user setuptools
pip3 install --user wheel
# Python 2
pip install --user setuptools
pip install --user wheel

Please, leave only the part for Python3. For windows we only provide support for Python3, not for Python2.

Copy link
Contributor Author

@corkyw10 corkyw10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 6 files reviewed, 2 unresolved discussions (waiting on @corkyw10 and @joel-mb)


Docs/build_linux.md, line 250 at r9 (raw file):

Previously, joel-mb wrote…
>This `.whl` file cannot be distributed as it is built specifically for your OS. To build a `.whl` for distribution, you can target a different system by using the following flag:
>- `make PythonAPI --target-wheel-platform="<platform>"` will build and repair the `.whl` file using `auditwheel` for a specific [platform](https://github.com/pypa/manylinux). External shared dependencies will be included in the file. The packaged version of the CARLA `.whl` targets the `manylinux_2_27_x86_64` platform.

Please, remove this part.

Done.


Docs/build_windows.md, line 50 at r9 (raw file):

Previously, joel-mb wrote…
Starting with CARLA 0.9.12, users have the option to install the CARLA Python API using `pip` or `pip3`. Version 20.3 or higher is required. To check if you have a suitable version, run the following command:
```sh
# For Python 3
pip3 -V
# For Python 2
pip -V

If you need to upgrade:

# For Python 3
pip3 install --upgrade pip
# For Python 2
pip install --upgrade pip

You must install the following Python dependencies:

# Python 3
pip3 install --user setuptools
pip3 install --user wheel
# Python 2
pip install --user setuptools
pip install --user wheel

Please, leave only the part for Python3. For windows we only provide support for Python3, not for Python2.

Done.

Copy link
Contributor

@joel-mb joel-mb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 6 files reviewed, all discussions resolved (waiting on @corkyw10 and @joel-mb)

@bernatx bernatx merged commit d4ead18 into dev Jul 30, 2021
@delete-merged-branch delete-merged-branch bot deleted the corkyw10/python-wheel-documentation branch July 30, 2021 07:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Requires changes/additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants