Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Try using bats for running lib/tests/modules.sh #185022

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

infinisil
Copy link
Member

Description of changes

This is an initial attempt at using bats to run lib/tests/modules.sh (but hopefully more in the future). By using bats we don't have to hand-roll our testing setup, it can take care of things like capturing exit status, command output, counting and reporting test failures. It also allows easily restructuring tests into a directory structure instead of having them all in a single file. It can also give names to tests.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.11 Release Notes (or backporting 22.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

- init bats-assert 2.0.0
- init bats-support 0.3.0
- init bats-file 0.3.0
Adds a `bats.withLibraries (p: [ ... ])` function, which creates a
`bats` wrapper where the `BATS_LIB_PATH` environment variable contains
fallbacks for the given list of libraries.

This allows to e.g. use the `bats-assert` library (which itself requires the
`bats-support` library) with

    bats.withLibraries (p: [ p.bats-support p.bats-assert ])

In a `.bats` file you can then call `bats_load_library` [1] to load the
libraries in the `setup()` function:

    setup() {
      bats_load_library bats-support
      bats_load_library bats-assert
    }

[1]: https://bats-core.readthedocs.io/en/stable/writing-tests.html?highlight=library#bats-load-library-load-system-wide-libraries
@abathur
Copy link
Member

abathur commented Aug 9, 2022

I have a little custom code in resholve's bats helpers to write test cases like these:

@test "allow (but do not parse) --fake 'source:path'" {
  require <({
    status 5
    line 3 ends "Unexpected absolute source path."
  })
} <<CASES
resholve --interpreter $INTERP --fake 'source:/not/a/real/script' < absolute_source.sh
CASES

@test "allow (*and* do not parse) --keep 'source:path' + --fake 'source:path'" {
  require <({
    status 0
    line 2 equals "source /not/a/real/script"
  })
} <<CASES
resholve --interpreter $INTERP --keep 'source:/not/a/real/script' --fake 'source:/not/a/real/script' < absolute_source.sh
resholve --interpreter $INTERP --keep '.:/not/a/real/script' --fake 'source:/not/a/real/script' < absolute_source.sh
CASES

@test "objects to unexempted tilde executable paths" {
  require <({
    status 7
    line -1 equals "[ stdinNone ]:2: Can't resolve dynamic command"
  })
} <<CASES
resholve --interpreter $INTERP < $PWD/tests/tilde_dynamic_pipeline.sh
CASES

I don't see any material improvements over what you're already getting from the assert library so far (and I haven't tried to support regex), but I could look at splitting it out if it's meaningful.

@infinisil infinisil added the 6.topic: module system About "NixOS" module system internals label Sep 28, 2023
@wegank wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: module system About "NixOS" module system internals 10.rebuild-darwin: 0 10.rebuild-linux: 0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants