Skip to content
This repository has been archived by the owner on Dec 16, 2022. It is now read-only.

Docker builds for other CUDA versions, improve CI #4796

Merged
merged 10 commits into from
Nov 16, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Docker builds for other CUDA versions
  • Loading branch information
epwalsh committed Nov 14, 2020
commit 925aab43f17f1256ca1986554baf23ba4dba28c5
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- Added Docker builds for other torch-supported versions of CUDA.
- Adds [`allennlp-semparse`](https://github.com/allenai/allennlp-semparse) as an official, default plugin.

### Fixed
Expand Down
7 changes: 6 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,14 @@ LABEL com.nvidia.volumes.needed="nvidia_driver"

WORKDIR /stage/allennlp

# Install torch first. This build arg should be in the from of a version requirement,
# like '==1.7' or '==1.7+cu102'.
ARG TORCH
RUN pip install --no-cache-dir torch${TORCH} -f https://download.pytorch.org/whl/torch_stable.html

# Install the wheel of AllenNLP.
COPY dist dist/
RUN pip install $(ls dist/*.whl)
RUN pip install --no-cache-dir $(ls dist/*.whl)
# TODO(epwalsh): In PyTorch 1.7, dataclasses is an unconditional dependency, when it should
# only be a conditional dependency for Python < 3.7.
# This has been fixed on PyTorch master branch, so we should be able to
Expand Down
5 changes: 5 additions & 0 deletions Dockerfile.test
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@ LABEL com.nvidia.volumes.needed="nvidia_driver"

WORKDIR /stage/allennlp

# Install torch first. This build arg should be in the from of a version requirement,
# like '==1.7' or '==1.7+cu102'.
ARG TORCH
RUN pip install --no-cache-dir torch${TORCH} -f https://download.pytorch.org/whl/torch_stable.html

# Installing AllenNLP's dependencies is the most time-consuming part of building
# this Docker image, so we make use of layer caching here by adding the minimal files
# necessary to install the dependencies. Since most of the dependencies are defined
Expand Down
15 changes: 11 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ MD_DOCS_EXTRAS = $(addprefix $(MD_DOCS_ROOT),README.md CHANGELOG.md CONTRIBUTING
DOCKER_TAG = latest
DOCKER_IMAGE_NAME = allennlp/allennlp:$(DOCKER_TAG)
DOCKER_TEST_IMAGE_NAME = allennlp/test:$(DOCKER_TAG)
# Our self-hosted runner currently has CUDA 11.0.
DOCKER_TEST_TORCH_VERSION = '==1.7.0+cu110'
DOCKER_RUN_CMD = docker run --rm \
-v $$HOME/.allennlp:/root/.allennlp \
-v $$HOME/.cache/torch:/root/.cache/torch \
Expand Down Expand Up @@ -139,17 +141,22 @@ clean :
.PHONY : docker-image
docker-image :
docker build \
--pull \
-f Dockerfile \
-t $(DOCKER_IMAGE_NAME) .
--pull \
-f Dockerfile \
--build-arg TORCH=$(DOCKER_TORCH_VERSION) \
-t $(DOCKER_IMAGE_NAME) .

.PHONY : docker-run
docker-run :
$(DOCKER_RUN_CMD) $(DOCKER_IMAGE_NAME) $(ARGS)

.PHONY : docker-test-image
docker-test-image :
docker build --pull -f Dockerfile.test -t $(DOCKER_TEST_IMAGE_NAME) .
docker build \
--pull \
-f Dockerfile.test \
--build-arg TORCH=$(DOCKER_TEST_TORCH_VERSION) \
-t $(DOCKER_TEST_IMAGE_NAME) .

.PHONY : docker-test-run
docker-test-run :
Expand Down