Skip to content

Commit

Permalink
ci: Test pathogen repo CI builds with the final packages
Browse files Browse the repository at this point in the history
[ Commit message based on that of 12000a20 in nextstrain/docker-base.¹
  Code changes also based on that commit, plus subsequent commits.² ]

A useful check for if new packages will break our pathogen builds.

I included all pathogen repos that already use our pathogen-repo-ci
reusable workflow.  It should be minimal effort to maintain this list
over time—I expect it to only grow—but perhaps in the future we will
want to abstract it out into a shared list of known pathogen repos.

I don't like that we have to copy the build-args for a few of the repos
here since it'll be easy for this copy to diverge from the repo's
authoritative build-args, but it's necessary for now.  Over time as we
work towards increased automation of pathogen builds, I think we can get
rid of this build-args copy by further standardizing how each repo
configures itself for automation.  For example, instead of specifying
build-args in a repo's CI workflow, the args for CI could be stored in a
broader workflow metadata file (e.g. nextstrain-workflow.yaml) read by
pathogen-repo-ci, or defined by some other convention.

An alternative to directly running pathogen-repo-ci against each repo
here would be instead triggering the CI workflows themselves within each
repo.  The downside to that is it would divorce the outcomes of those
workflows from this one and render them not visible from PRs in this
repo.  It would also require updates to each repo to support triggering
and passing in of additional parameters (i.e. for the package).  And
finally those CI workflows sometimes run other jobs, like linting and
other integration tests (e.g. with Cram), that aren't always necessary
to run with a new package.

Related-to: <nextstrain/docker-base#148>
Related-to: <nextstrain/docker-base#150>
Related-to: <nextstrain/docker-base#151>

¹ <nextstrain/docker-base@12000a20>
² <nextstrain/docker-base@bc22a0bc>
  <nextstrain/docker-base@0a20a474>
  <nextstrain/docker-base@75254e92>
  • Loading branch information
tsibley committed May 11, 2023
1 parent 560e502 commit a9442fb
Showing 1 changed file with 38 additions and 8 deletions.
46 changes: 38 additions & 8 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,13 @@ jobs:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
- id: set-version
run: echo "version=$(./devel/generate-version)" | tee -a "$GITHUB_OUTPUT"
- id: set
run: |
echo "version=$(./devel/generate-version)" | tee -a "$GITHUB_OUTPUT"
echo "label=$(./devel/label-for-ref "$GITHUB_REF")" | tee -a "$GITHUB_OUTPUT"
outputs:
version: ${{ steps.set-version.outputs.version }}
version: ${{ steps.set.outputs.version }}
label: ${{ steps.set.outputs.label }}

build-and-test:
needs: generate-version
Expand Down Expand Up @@ -78,8 +81,6 @@ jobs:
./devel/diff-pkgs nextstrain-base-*.conda build/locked/*/nextstrain-base-*.conda \
> "$GITHUB_STEP_SUMMARY"
# XXX TODO: Install Nextstrain CLI (standalone), then run test build.

release:
needs:
- generate-version
Expand All @@ -98,12 +99,10 @@ jobs:

- run: ./devel/setup

- name: Set LABEL
run: echo LABEL="$(./devel/label-for-ref "$GITHUB_REF")" | tee -a "$GITHUB_ENV"

- run: ./devel/upload
env:
ANACONDA_TOKEN: ${{ secrets.ANACONDA_TOKEN }}
LABEL: ${{ needs.generate-version.outputs.label }}

- name: Generate summary
run: |
Expand All @@ -113,3 +112,34 @@ jobs:
> "$GITHUB_STEP_SUMMARY"
env:
VERSION: ${{ needs.generate-version.outputs.version }}
LABEL: ${{ needs.generate-version.outputs.label }}

# Run pathogen repo CI builds with the final packages
test-pathogen-repo-ci:
needs:
- generate-version
- release
strategy:
# XXX TODO: Test on multiple platforms (os, maybe arch) via the matrix too
matrix:
include:
- { pathogen: avian-flu, build-args: auspice/flu_avian_h5n1_ha.json }
- { pathogen: ebola }
- { pathogen: lassa }
- { pathogen: monkeypox }
- { pathogen: mumps }
- { pathogen: ncov, build-args: all_regions -j 2 --profile nextstrain_profiles/nextstrain-ci }
- { pathogen: seasonal-flu, build-args: --configfile profiles/ci/builds.yaml -p }
- { pathogen: zika }
name: test-pathogen-repo-ci (${{ matrix.pathogen }})
uses: nextstrain/.github/.github/workflows/pathogen-repo-ci.yaml@master
with:
repo: nextstrain/${{ matrix.pathogen }}
build-args: ${{ matrix.build-args }}
runtimes: |
- conda
env: |
NEXTSTRAIN_CONDA_CHANNEL: nextstrain/label/${{ needs.generate-version.outputs.label }}
NEXTSTRAIN_CONDA_BASE_PACKAGE: nextstrain-base ==${{ needs.generate-version.outputs.version }}
artifact-name: ${{ matrix.pathogen }}-outputs
continue-on-error: true

0 comments on commit a9442fb

Please sign in to comment.