Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update GitHub actions and Python Dependencies #1821

Merged
merged 36 commits into from
Jan 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
5d1d101
Bump actions/setup-node from 3.5.1 to 3.6.0
dependabot[bot] Jan 19, 2023
0dc948e
Bump actions/checkout from 3.2.0 to 3.3.0
dependabot[bot] Jan 19, 2023
9e5d0cb
Bump actions/upload-artifact from 2.3.1 to 3.1.2
dependabot[bot] Jan 19, 2023
3a39ea7
Bump ossf/scorecard-action from 2.0.6 to 2.1.2
dependabot[bot] Jan 19, 2023
f68328b
Bump python from 3.10 to 3.11
dependabot[bot] Jan 19, 2023
3657a0d
Bump docker/build-push-action from 3.2.0 to 3.3.0
dependabot[bot] Jan 19, 2023
3ba5b19
Add api.securityscorecards.dev to allowed-endpoints
jmgrady Jan 19, 2023
e4a4d93
Merge remote-tracking branch 'origin/dependabot/github_actions/action…
jmgrady Jan 19, 2023
436e2dd
Merge remote-tracking branch 'origin/dependabot/github_actions/action…
jmgrady Jan 19, 2023
16ed1ac
Merge remote-tracking branch 'origin/dependabot/github_actions/action…
jmgrady Jan 19, 2023
0c89591
Update Python dependencies
jmgrady Jan 19, 2023
553649d
Remove *.data.mcr.microsoft.com endpoints
jmgrady Jan 20, 2023
83b7274
Print debugging info
jmgrady Jan 20, 2023
b607f6d
Test with bitnami images
jmgrady Jan 20, 2023
1013841
Add docker hub endpoints
jmgrady Jan 20, 2023
2c3836e
Fix image name
jmgrady Jan 20, 2023
f0c2288
Add Cloudflare endpoint for Docker
jmgrady Jan 20, 2023
b19139c
Add allowed endpoint
jmgrady Jan 20, 2023
dd00ed3
Set egress-policy to audit when building the backend image
jmgrady Jan 20, 2023
a248da0
Update python dependencies for deployment image
jmgrady Jan 20, 2023
d8ca98f
Merge remote-tracking branch 'origin/dependabot/github_actions/docker…
jmgrady Jan 20, 2023
1792af6
Merge remote-tracking branch 'origin/dependabot/github_actions/ossf/s…
jmgrady Jan 20, 2023
dd9b0e6
Merge branch 'update-gh-actions' into investigate-backend-build
jmgrady Jan 20, 2023
177a688
Merge remote-tracking branch 'origin/dependabot/docker/python-3.11' i…
jmgrady Jan 21, 2023
879206f
Merge remote-tracking branch 'origin/dependabot/docker/python-3.11' i…
jmgrady Jan 21, 2023
a084b9e
Merge branch 'investigate-backend-build' into update-gh-actions
jmgrady Jan 23, 2023
8535c44
Remove debugging aids
jmgrady Jan 23, 2023
b95eb0c
Update deployment build
jmgrady Jan 23, 2023
7447c13
update test YAML
jmgrady Jan 23, 2023
2b48c58
Add endpoint for api.github.com
jmgrady Jan 23, 2023
90462f3
Add ubuntu endpoints for maintenance build
jmgrady Jan 23, 2023
9a4703d
Remove test triggers & guards
jmgrady Jan 23, 2023
d7e68ae
Pin Python image in frontend build
jmgrady Jan 23, 2023
f455719
Pin Bitnami images
jmgrady Jan 23, 2023
e1807c5
Restrict permissions for Deploy QA & Deploy Release actions
jmgrady Jan 23, 2023
832dd53
Revert to running on push to master only
jmgrady Jan 23, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions .github/workflows/backend.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
github.com:443
storage.googleapis.com:443
uploader.codecov.io:443
- uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
- name: Setup dotnet
uses: actions/setup-dotnet@607fce577a46308457984d59e4954e075820f10a # v3.0.3
with:
Expand Down Expand Up @@ -79,7 +79,7 @@ jobs:
github.com:443

- name: Checkout repository
uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
# Manually install .NET to work around:
# https://github.com/github/codeql-action/issues/757
- name: Setup .NET
Expand All @@ -93,7 +93,7 @@ jobs:
- name: Autobuild
uses: github/codeql-action/autobuild@959cbb7472c4d4ad70cdfe6f4976053fe48ab394 # v2.1.37
- name: Upload artifacts if build failed
uses: actions/upload-artifact@82c141cc518b40d92cc801eee768e7aafc9c2fa2 # v2.3.1
uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # v3.1.2
if: ${{ failure() }}
with:
name: tracer-logs
Expand All @@ -103,7 +103,7 @@ jobs:

docker_build:
runs-on: ubuntu-22.04
if: ${{ github.event.type }} == "PullRequest"
# if: ${{ github.event.type }} == "PullRequest"
steps:
# For subfolders, currently a full checkout is required.
# See: https://github.com/marketplace/actions/build-and-push-docker-images#path-context
Expand All @@ -117,12 +117,12 @@ jobs:
dc.services.visualstudio.com:443
deb.debian.org:80
github.com:443
mcr.microsoft.com:443
eastus.data.mcr.microsoft.com:443
centralus.data.mcr.microsoft.com:443
westcentralus.data.mcr.microsoft.com:443
westus.data.mcr.microsoft.com:443
- uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
docker.io:443
auth.docker.io:443
registry-1.docker.io:443
production.cloudflare.docker.com
security.debian.org
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
with:
fetch-depth: 0
- name: Build backend
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ jobs:
egress-policy: block
allowed-endpoints: ${{ matrix.language.endpoints }}
- name: Checkout repository
uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/combine_deploy_image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:
username: ${{ secrets.AWS_ACCESS_KEY_ID }}
password: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
- name: Build combine_deploy
uses: docker/build-push-action@c56af957549030174b10d6867f20e78cfd7debc5 # v3.2.0
uses: docker/build-push-action@37abcedcc1da61a57767b7588cb9d03eb57e28b3 # v3.3.0
with:
context: "{{defaultContext}}:deploy"
push: true
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/database.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
with:
egress-policy: audit # TODO: change to 'egress-policy: block' after couple of runs

- uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
with:
fetch-depth: 0
- name: Build database image
Expand Down
39 changes: 21 additions & 18 deletions .github/workflows/deploy_qa.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,11 @@ name: "Deploy Update to QA Server"

on:
push:
branches: [master]
branches:
- master

permissions:
contents: read

jobs:
build:
Expand All @@ -22,26 +26,25 @@ jobs:
allowed-endpoints: >
${{ secrets.AWS_ACCOUNT }}.dkr.ecr.${{ secrets.AWS_DEFAULT_REGION }}.amazonaws.com
api.ecr.${{ secrets.AWS_DEFAULT_REGION }}.amazonaws.com:443
auth.docker.io:443
files.pythonhosted.org:443
github.com:443
sts.${{ secrets.AWS_DEFAULT_REGION }}.amazonaws.com:443
dotnetcli.azureedge.net:443
codecov.io:443
uploader.codecov.io:443
production.cloudflare.docker.com:443
pypi.org:443
docker.io:443
auth.docker.io:443
registry-1.docker.io:443
github.com:443
api.github.com:443
storage.googleapis.com:443
registry.npmjs.org:443
sts.${{ secrets.AWS_DEFAULT_REGION }}.amazonaws.com:443
api.nuget.org:443
pypi.org:443
files.pythonhosted.org:443
security.ubuntu.com:443
archive.ubuntu.com:443
dc.services.visualstudio.com:443
deb.debian.org:80
mcr.microsoft.com:443
eastus.data.mcr.microsoft.com:443
centralus.data.mcr.microsoft.com:443
westcentralus.data.mcr.microsoft.com:443
westus.data.mcr.microsoft.com:443
archive.ubuntu.com:80
security.ubuntu.com:80
api.github.com:443
- uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
with:
fetch-depth: 0
- name: Build The Combine
Expand Down Expand Up @@ -69,7 +72,7 @@ jobs:
api.ecr.${{ secrets.AWS_DEFAULT_REGION }}.amazonaws.com:443
github.com:443
sts.${{ secrets.AWS_DEFAULT_REGION }}.amazonaws.com:443
- uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@97271860067ec931c45b8d104fbf0d15954ab85c # v1-node16
with:
Expand All @@ -82,7 +85,7 @@ jobs:
needs: build
runs-on: [self-hosted, thecombine]
steps:
- uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
- name: Deploy The Combine Update
uses: ./.github/actions/combine-deploy-update
with:
Expand Down
7 changes: 5 additions & 2 deletions .github/workflows/deploy_release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ on:
branches: [master]
types: [published]

permissions:
contents: read

jobs:
build:
strategy:
Expand All @@ -19,7 +22,7 @@ jobs:
with:
egress-policy: audit # TODO: change to 'egress-policy: block' after couple of runs

- uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
- name: Build The Combine
id: build_combine
uses: ./.github/actions/combine-build
Expand All @@ -38,7 +41,7 @@ jobs:
needs: build
runs-on: [self-hosted, thecombine]
steps:
- uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
with:
fetch-depth: 0
- name: Deploy The Combine Update
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/frontend.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@ jobs:
registry.npmjs.org:443
storage.googleapis.com:443
uploader.codecov.io:443
- uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@8c91899e586c5b171469028077307d293428b516 # v3.5.1
uses: actions/setup-node@64ed1c7eab4cce3362f8c340dee64e5eaeef8f7c # v3.6.0
with:
node-version: ${{ matrix.node-version }}
- run: npm ci
Expand Down Expand Up @@ -68,7 +68,7 @@ jobs:
pypi.org:443
registry-1.docker.io:443
registry.npmjs.org:443
- uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
with:
fetch-depth: 0
- name: Build frontend
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/maintenance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
pypi.org:443
registry-1.docker.io:443
security.ubuntu.com:80
- uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
with:
fetch-depth: 0
- name: Build maintenance image
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
files.pythonhosted.org:443
github.com:443
pypi.org:443
- uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
- uses: actions/setup-python@5ccb29d8773c3f3f653e1705f474dfaa8a06a912 # v4.4.0
with:
python-version: 3.9
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:
files.pythonhosted.org:443
github.com:443
pypi.org:443
- uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@5ccb29d8773c3f3f653e1705f474dfaa8a06a912 # v4.4.0
with:
Expand Down
9 changes: 4 additions & 5 deletions .github/workflows/scorecards.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,6 @@ on:
- cron: "18 6 * * 0"
push:
branches: [master]
pull_request:
branches: [master]

# Declare default permissions as read only.
permissions: read-all
Expand Down Expand Up @@ -49,13 +47,14 @@ jobs:
sigstore-tuf-root.storage.googleapis.com:443
auth.docker.io:443
rekor.sigstore.dev:443
api.securityscorecards.dev
- name: "Checkout code"
uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8 # v3.1.0
uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
with:
persist-credentials: false

- name: "Run analysis"
uses: ossf/scorecard-action@99c53751e09b9529366343771cc321ec74e9bd3d # v2.0.6
uses: ossf/scorecard-action@e38b1902ae4f44df626f11ba0734b14fb91f8f86 # v2.1.2
with:
results_file: results.sarif
results_format: sarif
Expand All @@ -77,7 +76,7 @@ jobs:
# Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF
# format to the repository Actions tab.
- name: "Upload artifact"
uses: actions/upload-artifact@3cea5372237819ed00197afe530f5a7ea3e805c8 # v3.1.0
uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # v3.1.2
with:
name: SARIF file
path: results.sarif
Expand Down
4 changes: 2 additions & 2 deletions Backend/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Docker multi-stage build.
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS builder
FROM bitnami/dotnet-sdk:6@sha256:b8426ee39b7ec43d905d3b198f7819552c791472485fae95275e3832ab251480 AS builder
WORKDIR /app

# Copy csproj and restore (fetch dependencies) as distinct layers.
Expand All @@ -11,7 +11,7 @@ COPY . ./
RUN dotnet publish -c Release -o build

# Build runtime image.
FROM mcr.microsoft.com/dotnet/aspnet:6.0
FROM bitnami/aspnet-core@sha256:1fa0ee4f642f484066b236a4cb8629cafbbc21c485939b0da52f6d0e9b36f3f7

ENV ASPNETCORE_URLS=http://+:5000
ENV COMBINE_IS_IN_CONTAINER=1
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# User guide build environment.
FROM python:3.10 AS user_guide_builder
FROM python:3.11@sha256:a3c0c6766535f85f18e7304d3a0111de5208d73935bcf1b024217005ad5ce195 AS user_guide_builder

ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
Expand Down
36 changes: 17 additions & 19 deletions deploy/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,28 +1,28 @@
#
# This file is autogenerated by pip-compile with python 3.10
# To update, run:
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
# pip-compile requirements.in
#
ansible==6.5.0
ansible==7.1.0
# via -r requirements.in
ansible-core==2.13.5
ansible-core==2.14.1
# via ansible
cachetools==5.2.0
cachetools==5.2.1
# via google-auth
certifi==2022.9.24
certifi==2022.12.7
# via
# kubernetes
# requests
cffi==1.15.1
# via cryptography
charset-normalizer==2.1.1
charset-normalizer==3.0.1
# via requests
cryptography==38.0.1
cryptography==39.0.0
# via
# ansible-core
# pyopenssl
google-auth==2.12.0
google-auth==2.16.0
# via kubernetes
idna==3.4
# via requests
Expand All @@ -33,13 +33,13 @@ jinja2==3.1.2
# jinja2-base64-filters
jinja2-base64-filters==0.1.4
# via -r requirements.in
kubernetes==24.2.0
kubernetes==25.3.0
# via -r requirements.in
markupsafe==2.1.1
markupsafe==2.1.2
# via jinja2
oauthlib==3.2.1
oauthlib==3.2.2
# via requests-oauthlib
packaging==21.3
packaging==23.0
# via ansible-core
pyasn1==0.4.8
# via
Expand All @@ -49,18 +49,16 @@ pyasn1-modules==0.2.8
# via google-auth
pycparser==2.21
# via cffi
pyopenssl==22.1.0
pyopenssl==23.0.0
# via -r requirements.in
pyparsing==3.0.9
# via packaging
python-dateutil==2.8.2
# via kubernetes
pyyaml==6.0
# via
# -r requirements.in
# ansible-core
# kubernetes
requests==2.28.1
requests==2.28.2
# via
# kubernetes
# requests-oauthlib
Expand All @@ -75,11 +73,11 @@ six==1.16.0
# google-auth
# kubernetes
# python-dateutil
urllib3==1.26.12
urllib3==1.26.14
# via
# kubernetes
# requests
websocket-client==1.4.1
websocket-client==1.4.2
# via kubernetes

# The following packages are considered to be unsafe in a requirements file:
Expand Down
2 changes: 0 additions & 2 deletions deploy/scripts/build.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,6 @@ def start_next(self) -> bool:
running or if the queue is empty.
"""
if self.curr_job is not None and self.curr_job.poll() is None:
logging.debug(f"{self.name}.start_next(): called while job is still running.")
return False
if len(self.job_list) > 0:
next_job = self.job_list.pop(0)
Expand Down Expand Up @@ -95,7 +94,6 @@ def check_jobs(self) -> JobStatus:
if self.curr_job is not None:
# See if the job is still running
if self.curr_job.poll() is None:
logging.debug(f"{self.name} job is running.")
return self.status
# Current job has finished
if self.curr_job.returncode == 0:
Expand Down
Loading