Skip to content
This repository has been archived by the owner on Dec 16, 2022. It is now read-only.

Docker builds for other CUDA versions, improve CI #4796

Merged
merged 10 commits into from
Nov 16, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
clean up
  • Loading branch information
epwalsh committed Nov 16, 2020
commit b343c6f3f0152ac3024dac61d60c17a01ca32ff0
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ MD_DOCS_EXTRAS = $(addprefix $(MD_DOCS_ROOT),README.md CHANGELOG.md CONTRIBUTING
DOCKER_TAG = latest
DOCKER_IMAGE_NAME = allennlp/allennlp:$(DOCKER_TAG)
DOCKER_TEST_IMAGE_NAME = allennlp/test:$(DOCKER_TAG)
DOCKER_TORCH_VERSION = 'torch==1.7.0'
# Our self-hosted runner currently has CUDA 11.0.
DOCKER_TEST_TORCH_VERSION = 'torch==1.7.0+cu110 -f https://download.pytorch.org/whl/torch_stable.html'
DOCKER_RUN_CMD = docker run --rm \
Expand Down
61 changes: 21 additions & 40 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,26 @@ docker run --rm --gpus all -v $HOME/.allennlp:/root/.allennlp allennlp/allennlp:

If you don't have GPUs available, just omit the `--gpus all` flag.

#### Building your own Docker image

For various reasons you may need to create your own AllenNLP Docker image, such as if you need a different version
of PyTorch. To do so, just run `make docker-image` from the root of your local clone of AllenNLP.

By default this builds an image with the tag `allennlp/allennlp`, but you can change this to anything you want
by setting the `DOCKER_TAG` flag when you call `make`. For example,
`make docker-image DOCKER_TAG=my-allennlp`.

If you want to use a different version of PyTorch, set the flag `DOCKER_TORCH_VERSION` to something like
`torch==1.7.0` or `torch==1.7.0+cu110 -f https://download.pytorch.org/whl/torch_stable.html`.
The value of this flag will passed directly to `pip install`.

After building the image you should be able to see it listed by running `docker images allennlp`.

```
REPOSITORY TAG IMAGE ID CREATED SIZE
allennlp/allennlp latest b66aee6cb593 5 minutes ago 2.38GB
```

### Installing from source

You can also install AllenNLP by cloning our git repository:
Expand All @@ -233,7 +253,7 @@ You can also install AllenNLP by cloning our git repository:
git clone https://github.com/allenai/allennlp.git
```

Create a Python 3.7 virtual environment, and install AllenNLP in `editable` mode by running:
Create a Python 3.7 or 3.8 virtual environment, and install AllenNLP in `editable` mode by running:

```bash
pip install --editable .
Expand All @@ -254,45 +274,6 @@ with the `allennlp` command (whether you installed from `pip` or from source).
`allennlp` has various subcommands such as `train`, `evaluate`, and `predict`.
To see the full usage information, run `allennlp --help`.

## Docker images

AllenNLP releases Docker images to [Docker Hub](https://hub.docker.com/r/allennlp/) for each release. For information on how to run these releases, see [Installing using Docker](#installing-using-docker).

### Building a Docker image
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did this go away?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just moved it into the "Installing via Docker" section.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You did 🙈


For various reasons you may need to create your own AllenNLP Docker image.
The same image can be used either with a CPU or a GPU.

First, you need to [install Docker](https://www.docker.com/get-started).
Then you will need a wheel of allennlp in the `dist/` directory.
You can either obtain a pre-built wheel from a PyPI release or build a new wheel from
source.

PyPI release wheels can be downloaded by going to https://pypi.org/project/allennlp/#history,
clicking on the desired release, and then clicking "Download files" in the left sidebar.
After downloading, make you sure you put the wheel in the `dist/` directory
(which may not exist if you haven't built a wheel from source yet).

To build a wheel from source, just run `python setup.py wheel`.

*Before building the image, make sure you only have one wheel in the `dist/` directory.*

Once you have your wheel, run `make docker-image`. By default this builds an image
with the tag `allennlp/allennlp`. You can change this to anything you want
by setting the `DOCKER_TAG` flag when you call `make`. For example,
`make docker-image DOCKER_TAG=my-allennlp`.

You should now be able to see this image listed by running `docker images allennlp`.

```
REPOSITORY TAG IMAGE ID CREATED SIZE
allennlp/allennlp latest b66aee6cb593 5 minutes ago 2.38GB
```

### Running the Docker image

You can run the image with `docker run --rm -it allennlp/allennlp:latest`. The `--rm` flag cleans up the image on exit and the `-it` flags make the session interactive so you can use the bash shell the Docker image starts.

You can test your installation by running `allennlp test-install`.

## Issues
Expand Down