[RFC 0095] Enable doCheck by default

rfcs/0095-enable-docheck-by-default.md

@@ -0,0 +1,69 @@
+---
+feature: enable-docheck-by-default
+start-date: 2021-06-24
+author: gytis-ivaskevicius
+co-authors:
+shepherd-team:
+shepherd-leader:
+related-issues:
+---
+
+# Summary
+[summary]: #summary
+
+Enable `doCheck` by default when using `stdenv.mkDerivation` function.
+
+# Motivation
+[motivation]: #motivation
+
+I believe that an additional quality gate would be beneficial to the derivations build process.
+
+Resolution of this RFC is expected to remove [these comments](https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/generic/make-derivation.nix#L61-L67)
+either by enabling checkPhase by default or rejecting this RFC.
+
+# Detailed design
+[design]: #detailed-design
+
+The basic idea is quite simple
+- New `doCheck` semantic should be implemented.
+- By default `doCheck` option should be enabled as long as `stdenv.hostPlatform == stdenv.buildPlatform`.
+- Non-reproducible test prevention should be implemented.
+- All failing packages should be fixed or updated with `doCheck = false;`
+
+**New doCheck semantics:**
+Instead of boolean it should accept these values:
+- true - Run both `checkPhase` and `installCheckPhase`.
+- false - Do not execute `checkPhase` nor `installCheckPhase`.
+- "checkPhase" - Execute `checkPhase` only.
+- "installCheckPhase" - Execute `installCheckPhase` only.
+- _Anything else_ - Same as `false` but also adds some verbosity which makes it
+  valuable if it has a comment or actual reason why the test fails. For example
+  "Requires X11 server" or "Requires network access".
+
+** Non-reproducible tests prevention:**
+There are multiple options. Here I am going to list a few:
+1. `chmod a-w -R /build`
+2. [User namespaces](https://lwn.net/Articles/532593/)
+3. Generate unique identifier from existing sources and compare it with
+   identifier generated after executing `checkPhase`. `exit 1` If values
+   mismatch. (Identifier can be generated by something simple like `du -s`)
+4. [unionfs](https://en.wikipedia.org/wiki/UnionFS)
+
+# Drawbacks
+[drawbacks]: #drawbacks
+
+- Increased build time.
+- Increases likelihood of failing derivations after a simple update.
+
+# Alternatives
+[alternatives]: #alternatives
+
+The impact of avoiding this is a lack of assurance of delivered package quality.
+
+# Unresolved questions
+[unresolved]: #unresolved-questions
+
+"Non-reproducible tests prevention" implementation is to be decided. I feel
+like `du -s` is the right way to go about it since it is simple/fast and I
+expect it to be quite reliable.
+