Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backup improvements #1005

Merged
merged 11 commits into from
Feb 11, 2021
Merged

Backup improvements #1005

merged 11 commits into from
Feb 11, 2021

Conversation

jmgrady
Copy link
Collaborator

@jmgrady jmgrady commented Feb 10, 2021

Implements the following improvements to the backup process for TheCombine:

  1. Before the backup of the backend, the process deletes ZIP files from project import/export operations that have not been accessed in the past day;
  2. Splits out common script variables into a bin/combine-env script (currently shared by the backup and restore processes)
  3. Pipes cron job output to rlogger so that the backup results are logged in /var/log/syslog
  4. Version of aws-cli is controlled by ./deploy/vars/packages.yml
  5. Removes shutdown/restart of backend and database containers as part of the backup.

This change is Reviewable

Move to backup time to accommodate west coast users.
Enable only installing/updating the backup configuration by specifying:
 -t backups
on ansible-playbook command line.
1. common combine-env script for setting common environment variables 
and functions
2. rename combine-cleanup to combine-clean-aws
3. send backup output to rsyslog instead of email (default)
4. remove import/export zip files before backup
5. remove shutdown/restart of backend and database for backup
@jmgrady jmgrady added deployment maintenance Issue that makes it difficult to maintain the software or to upgrade installations post-release. labels Feb 10, 2021
@jmgrady jmgrady self-assigned this Feb 10, 2021
Color coding shows up as escape characters in syslog
Copy link
Collaborator

@imnasnainaec imnasnainaec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 6 of 12 files at r1.
Reviewable status: 6 of 12 files reviewed, all discussions resolved (waiting on @jmgrady)


deploy/roles/combine_backup/defaults/main.yml, line 2 at r1 (raw file):

# backup_hour:  7
# backup_minute: 15
max_backups: 3

The hour in this comment doesn't match the 10 in deploy/group_vars/server/main.yaml.

Copy link
Collaborator Author

@jmgrady jmgrady left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 6 of 12 files reviewed, all discussions resolved (waiting on @jmgrady)


deploy/roles/combine_backup/defaults/main.yml, line 2 at r1 (raw file):

Previously, imnasnainaec (D. Ror.) wrote…
# backup_hour:  7
# backup_minute: 15
max_backups: 3

The hour in this comment doesn't match the 10 in deploy/group_vars/server/main.yaml.

The comment is intended to show the names of variables to be defined to enable the backups. I have added some text to explain this.

Copy link
Collaborator

@imnasnainaec imnasnainaec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 5 of 12 files at r1, 1 of 1 files at r2.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved

@jmgrady jmgrady merged commit 4605a4b into master Feb 11, 2021
@jmgrady jmgrady deleted the backup_improvements branch February 11, 2021 01:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deployment maintenance Issue that makes it difficult to maintain the software or to upgrade installations post-release.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants