treewide: sha512 → hash

[nbraud]

Description of changes

• Improve sha-to-hash.py to

  • handle multiple hash functions (currently SHA-256 and SHA-512),
  • automatically skip autogenerated files, based on heuristics.

• Replace sha512 with hash through nixpkgs, where appropriate.
  This does not include autogenerated files; the tools making those should be updated & rerun.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Fits CONTRIBUTING.md.

[nbraud]

Perfect issue number for this <3

[nbraud]

PS: Rebased to deal with two minor oopsies in the history:

• one small fix had been squashed in the wrong commit;
• one commit's message was obsolete, as I didn't add SHA-1 support in the end:
  SHA-1 is invalid in SRI hashes, even though nix accepts it; see Nix accepts invalid SRI hashes (MD5 and SHA-1) nix#8982.

[vcunat]

For reference, lots of discussion around this happened at NixOS/rfcs#131

[nbraud]

For reference, lots of discussion around this happened at NixOS/rfcs#131

Thanks for the pointer! It looks like the RFC was about automatically enforcing the use of hash, though?
This PR only affects what little manually-written Nix code uses sha512 (the diff only lists 38 changed hashes).
In particular, I'm very much not trying to move autogenerated files over, or even (yet) list all nix-generating tools that would need updating.

Overall, what motivated me to try and migrate most of nixpkgs over, is that I noticed some nix users seem to be confused by the sha256 and sha512 parameters, most likely because they were removed from the fetchers' docs (in favor of hash) but are still in widespread use.

[vcunat]

That sounds like it stems from a split in the community where to move on this topic (whether to prefer SRI and how strongly), which seems very much like that RFC.

[vcunat]

From what you say I'd think it would be better to document them. Even if they would be considered deprecated for new packages or something, in which case you just express that in those docs (though I'm not aware of such a decision so far).

[nbraud]

From what you say I'd think it would be better to document them.

Presumably, they were previously documented and were eventually removed. IDK which PR removed it, and for what reason, so I'm certainly not going to blindly revert that.

Even if they would be considered deprecated for new packages or something, in which case you just express that in those docs (though I'm not aware of such a decision so far).

IDK how normative that is, but Eelco Dolstra was already stating in 2020 that “nowadays it's better to use SRI hashes, which is what the new CLI defaults to.”

In general, I agree it makes sense to have a hash format that is preferred and tooling defaults to; especially if that format is an actual standard, so we interop with other software.

[nbraud]

That sounds like it stems from a split in the community where to move on this topic (whether to prefer SRI and how strongly), which seems very much like that RFC.

BTW, going by what's in nixpkgs, there doesn't seem to be that much of a split on how to represent sha512 hashes:

• 3 are in “nix32” format;
• 167 are hex-encoded... plus 10188 more in texlive/tlpdb.nix (which seems auto-generated)
• 13660 are in base64.

You can reproduce that with

set -A formats "hex" "[0-9A-Fa-f]{128}" "nix32" "[0-9a-fg-np-z]{103}" "base64" "(sha512-)?[A-Za-z0-9+/]{86}={2}"
for format regex in "${(@kv)formats}"; do
  echo -n "$format: "
  rg -c -g "*.nix" "['\"]${regex}['\"]" | cut -d: -f2 | paste -sd+ | bc
done

[nbraud]

Rebased to address merge conflict.

[ulrikstrid]

From the point of view of OCaml we have already been moving this way so this is fine. I have not reviewed the script(s).

[AndersonTorres]

I don't know about pretty printers in Python.
Besides, LGTM