Skip to content

Commit

Permalink
test: Run tests faster with nextest.
Browse files Browse the repository at this point in the history
> [`cargo-nextest`](https://nexte.st/index.html) is a next-generation
> test runner for Rust projects.

This patch installs and uses `nextest` to run our own tests.

Comparing `cargo test` and `cargo nextest` with hyperfine provides the
following results:

```sh
$ hyperfine 'cargo test --workspace' 'cargo nextest run --workspace && cargo test --doc'
Benchmark 1: cargo test --workspace
  Time (mean ± σ):     51.785 s ±  2.066 s    [User: 183.471 s, System: 10.563 s]
  Range (min … max):   49.151 s … 56.641 s    10 runs

Benchmark 2: cargo nextest run --workspace && cargo test --doc
  Time (mean ± σ):     44.556 s ±  0.894 s    [User: 192.213 s, System: 11.441 s]
  Range (min … max):   43.170 s … 45.762 s    10 runs
```

Benchmark 2 is 1.16 times faster than Benchmark 1.
  • Loading branch information
Hywan committed Jun 22, 2022
1 parent 87639a4 commit 399862d
Show file tree
Hide file tree
Showing 4 changed files with 20 additions and 5 deletions.
3 changes: 3 additions & 0 deletions .github/workflows/appservice.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,9 @@ jobs:
- name: Load cache
uses: Swatinem/rust-cache@v1

- name: Install nextest
uses: taiki-e/install-action@nextest

- name: Run checks
uses: actions-rs/cargo@v1
with:
Expand Down
9 changes: 8 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,9 @@ jobs:
- name: Load cache
uses: Swatinem/rust-cache@v1

- name: Install nextest
uses: taiki-e/install-action@nextest

- name: Test
uses: actions-rs/cargo@v1
with:
Expand Down Expand Up @@ -115,10 +118,14 @@ jobs:
- name: Load cache
uses: Swatinem/rust-cache@v1

- name: Install nextest
uses: taiki-e/install-action@nextest

- name: Test
uses: actions-rs/cargo@v1
with:
command: test
command: nextest
args: run

test-nodejs:
name: linux / node.js (${{ matrix.node-version }})
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/wasm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,9 @@ jobs:
- name: Load cache
uses: Swatinem/rust-cache@v1

- name: Install nextest
uses: taiki-e/install-action@nextest

- name: Rust Check
uses: actions-rs/cargo@v1
with:
Expand Down
10 changes: 6 additions & 4 deletions xtask/src/ci.rs
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,9 @@ fn run_feature_tests(cmd: Option<FeatureSet>) -> Result<()> {
]);

let run = |arg_set: &str| {
cmd!("rustup run stable cargo test -p matrix-sdk").args(arg_set.split_whitespace()).run()
cmd!("rustup run stable cargo nextest run -p matrix-sdk")
.args(arg_set.split_whitespace())
.run()
};

match cmd {
Expand All @@ -186,15 +188,15 @@ fn run_crypto_tests() -> Result<()> {
"rustup run stable cargo clippy -p matrix-sdk-crypto --features=backups_v1 -- -D warnings"
)
.run()?;
cmd!("rustup run stable cargo test -p matrix-sdk-crypto --features=backups_v1").run()?;
cmd!("rustup run stable cargo test -p matrix-sdk-crypto-ffi").run()?;
cmd!("rustup run stable cargo nextest run -p matrix-sdk-crypto --features=backups_v1").run()?;
cmd!("rustup run stable cargo nextest run -p matrix-sdk-crypto-ffi").run()?;

Ok(())
}

fn run_appservice_tests() -> Result<()> {
cmd!("rustup run stable cargo clippy -p matrix-sdk-appservice -- -D warnings").run()?;
cmd!("rustup run stable cargo test -p matrix-sdk-appservice").run()?;
cmd!("rustup run stable cargo nextest run -p matrix-sdk-appservice").run()?;

Ok(())
}
Expand Down

0 comments on commit 399862d

Please sign in to comment.