Run nixos tests for each PR

[DavHau]

Is your feature request related to a problem? Please describe.
When a PR breaks a nixos test, there is no obvious feedback about it and it doesn't block the merge.

Describe the solution you'd like
Provide fast feedback about any breaking test to the author of a PR as well as the reviewers.

Describe alternatives you've considered
No quick feedback about breaking nixos tests
-> reduced confidence in changes
-> increased overhead for reviewers
-> harder to track down cause once breakage gets discovered.

Additional context
What's a quick and easy way to get CI checks for the nixos tests?

Ideas:

• use hydra and connect to github somehow
• rent hosted third party github runners with KVM (not sure where)
• self host github runners
• use hercules (cc @roberth )
• use garnix
• use https://github.com/Mic92/buildbot-nix (cc @Mic92 )
• polish and use @Mic92's systemd-nspawn test driver instead of relying on KVM

Priorities

Add 👍 to issues you find important.

[Mic92]

Github's own runner now also supports kvm btw.

[DavHau]

Github's own runner now also supports kvm btw.

Nice, thanks for the hint. I just tried out running all the tests in a runner.
It seems to be unacceptably slow. I mean it already was before enabling the tests. Compiling nix alone takes around 15 minutes on the linux runner. What can we do to improve this situation?

[Mic92]

I don't think we can improve nix compilation time magically. Regarding nixos tests, I was expecting no significant slow down since download dependencies should be finished within the 15 minutes window. Can you check if /dev/kvm is present in the build environment?
There is the magic-nix-cache but it only improves build times if there are some custom packages or overrides that benefit from caching: https://github.com/Nix-Security-WG/nix-security-tracker/blob/cbd2abdef8ec77f0ac9d6ab7b5e5414d972c41c5/.github/workflows/checks.yaml#L12

[zowoq]

Also see the discussion in #7674.

[roberth]

An extra goal - perhaps separate issue? - is to include tests that Nixpkgs thinks are relevant.
We can configure the nix flake to run those, with a little bit of collaboration between the repos.