Managing Travis

Caching

Downloading dependencies such as Solr and Fedora each time for testing can take time. Also, Apache will throw Bandwidth Exceeded errors if you attempt to download them many times in the course of a day. We cache these and other downloads in order to prevent errors and speed-up the build process.

This happens in .travis.yml

cache:
  directories:
    - "dep_cache"
    - "travis_phantomjs"

We setup a directory on Travis called dep_cache and have special config files for solr_wrapper and fcrepo_wrapper that download their files to that directory. During the before_install of the Travis build, we ensure the directory is there (if it isn't already), list its contents to the log file, and copy over the Travis-specific config files.

The travis_phantomjs directory caches the phantomJS download. All of these directories are referenced in travis/test.sh and travis/coverage.sh.

The cache will already be present in the develop branch, and any new branches created should use the same cache. You can look at the log file for each build to verify the files are present.

Clearing the Cache

If we update the versions of Solr or Fedora, the wrapper gems should simply download new files and place them in the existing cache folder. The old files will remain until explicitly removed by us or Travis.

Stage Builds

The Travis build is broken up into three stages:

1. Rubocop
2. Test
3. Coverage

Each stage runs one or more jobs. If there are multiple jobs, they are run in parallel on separate instances. Test is the default stage and is always included, but because we want it to run in a specific order, we use the stages key to set the order:

stages:
  - rubocop
  - test
  - coverage

The first stage runs rubocop separately from the other builds because no special scripted setup is required. It runs fastest as a simple rake command:

  - stage: rubocop
  script: bundle exec rake scholarsphere:travis:rubocop

Test, the second stage, runs the travis/test.sh script for unit and feature tests. Both tests run in parallel using matrix expansion specified under jobs via the two script and env directives. Because test is the default, included stage, it is first in the include section even though it will be run as the second stage.

jobs:
  include:
    - script: ./travis/test.sh
      env: TEST_SUITE=feature
    - script: ./travis/test.sh
      env: TEST_SUITE=unit

The coverage stage runs last as a separate travis/coverage.sh script, uploading the coverage reports generated in the previous stage to Code Climate.

AWS

In order to send a complete coverage report to Code Climate, we need to combine the feature and unit test reports. Because files are not saved between stages and jobs, at the end of each test, the coverage report is uploaded to S3 for storage, then the coverage stage downloads both reports and sends a combined one to Code Climate.

The authentication to AWS is accomplished via encrypted environment variables.

Details of the AWS integration can be found in our local documentation.

Encrypted Environment Variables

Keys used by Travis to upload data to AWS and to send code reports to Code Climate are stored as encrypted variables using the travis gem. See https://docs.travis-ci.com/user/environment-variables/#Defining-encrypted-variables-in-.travis.yml for more information.

env:
  global:
    - secure: "xxxxxxxxxxx"
    - secure: "xxxxxxxxxxx"
    - secure: "xxxxxxxxxxx"

Travis decrypts each of these when it runs. You can see the decrypted variable name (but not the value!) in the log of the job.