Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run functional tests in Travis CI #753

Closed
huddlej opened this issue Oct 7, 2021 · 2 comments
Closed

Run functional tests in Travis CI #753

huddlej opened this issue Oct 7, 2021 · 2 comments
Labels
enhancement New feature or request

Comments

@huddlej
Copy link
Contributor

huddlej commented Oct 7, 2021

Context

We've started to write Cram-based functional tests for this workflow and associated scripts, but we do not currently run these tests as part of our standard Travis CI.

Description

Not all critical components of the workflow are tested by the minimal CI profile. We should run functional tests along with the current minimal Snakemake run as part of CI.

Possible solution

Naively, we could make this happen by installing Cram with pip and running Cram on all *.t files in the tests/ directory.

Comments in tests/different-inputs.t suggest that Cram should be run with cram --preserve-env tests/different-inputs.t.

In practice, the "different inputs" test takes >8 minutes to run, more than we typically prefer for CI tests. Maybe we could pare these tests down to more minimal versions that test the same logic?

@huddlej huddlej added the enhancement New feature or request label Oct 7, 2021
@jameshadfield
Copy link
Member

#671 started down this road, but using GitHub Actions, it'd be great to pick this up. (I'd prefer we move to one CI runner where possible, but that's part of a bigger discussion.)

@huddlej
Copy link
Contributor Author

huddlej commented Nov 25, 2021

This was mostly resolved by #767. How to handle the "different inputs" tests should be a separate issue.

@huddlej huddlej closed this as completed Nov 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants