std: implement the random feature

[joboet]

Implements the ACP rust-lang/libs-team#393.

API

This PR contains two minor deviations to the API proposed in the ACP (I can easily undo them if necessary):

• gen_bytes is renamed to fill_bytes
• DefaultRng is renamed to DefaultRandomSource

Most names in std do not contain abbreviations, and even the documentation proposed in the ACP describes gen_bytes with "Fill buf with random bytes". DefaultRandomSource mirrors DefaultHasher, both names contain the relevant trait.

Implementation

The implementation in this PR mostly reuses the old hash-map key generation code, but contains a few deviations:

• The Linux code was amended to poll /dev/random before reading /dev/urandom which is required to ensure that the entropy pool is initialized.
• The fallback code to /dev/urandom has been removed for FreeBSD and some other UNIX-like OSs, they now all use getentropy instead of getrandom. This function is always available on all of them (FreeBSD 12 is our minimum version, see the module documentation in sys/random/unix.rs for the relevant documentation) and has been included in the lastest version of the POSIX standard, issue 8, so using it whereever possible seems like a good idea, especially since NetBSD even indicates that getrandom might be removed in the future. Falling back is unnecessary as if getentropy fails, /dev/urandom will, too, as FreeBSD does not support retrieving insecure random data.

I'd appreciate help from all platform maintainers to help verify that all of these implementations are correct and use a suitable API.

Discussion about the API

I have some issues with the API myself, but to keep this PR focussed on discussion the actual implementation, I'd like to ask everyone to keep these discussions to the ACP issue.

[rustbot]

r? @cuviper

rustbot has assigned @cuviper.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[joboet]

While implementing, I couldn't help but notice that quite a lot of platforms warn against using their randomness syscalls for non-seed-generating purposes. For instance, the newly introduced POSIX function getentropy specifies that:

"The intended use of this function is to create a seed for other pseudo-random number generators."

My gut instinct is that people will attempt to use DefaultRandomSource for much more than that, meaning we expose platforms to loads of small random data request, which they might not be equipped to deal with. So perhaps we should use things like arc4random_buf instead and clarify that the random numbers are not the highest quality the system can provide (but good enough in practise, arc4random is still cryptographically secure)?

[joshtriplett]

No objection to renaming to DefaultRandomSource. I'd personally prefer not to rename gen_bytes to fill_bytes though. It's a random number generator, and if people do end up calling it directly I think that name would be more evocative. This isn't a blocker, though, and if others on libs-api want it renamed I won't object.

[the8472]

There are edge-cases where anything short of the OS entropy provider or cpu instructions¹ will not be suitable for cryptographic use. If it involves fork() or cloning VM snapshots you end up with reusing RNG state and thus potential key/nonce reuse which can be catastrophic for some cryptographic schemes. A libc implementation can guard against forking, but not against VM clones.

¹ and those have had bugs, like returning all-0 "entropy".

use things like arc4random_buf instead and clarify that the random numbers are not the highest quality the system can provide (but good enough in practise, arc4random is still cryptographically secure)?

Old implementations that actually use RC4 are broken. Newer ones that switch to modern stream ciphers should be secure except for the reuse issue.

[clarfonthey]

You mention discussing on the ACP, but the ACP has been approved and is closed. You should file a tracking issue and link that on the features you implement so we can better track design concerns there.

[joboet]

I'd personally prefer not to rename gen_bytes to fill_bytes though. It's a random number generator, and if people do end up calling it directly I think that name would be more evocative.

How about just generate, then? It's short, clear and not an abbreviation. Either way, I don't really mind.

[cuviper]

Given the continued discussion on the ACP(s), I think this should have an API reviewer.

r? libs-api

[bors]

☔ The latest upstream changes (presumably #129398) made this pull request unmergeable. Please resolve the merge conflicts.

[juntyr]

Will this type only ever be used for creating hash-map seeds?

As noted in #129402, a future wasip2 implementation might use wasi:random/insecure for best effort randomness and wasi:random/random for the default random source. However, wasip2 also has the specific wasi:random/insecure-seed interface which seems an even closer fit for generating hash map keys.

If this type would only ever be used for initialising hash map random keys, the wasip2 implementation could just use wasi:random/insecure-seed. However, if this type became more widely used inside std (or even exposed), it might need to fall back to wasi:random/insecure.

[joboet]

Fair point, I'll just reintroduce hashmap_random_seeds.

[joboet]

Rebased to reintroduce hashmap_random_keys, fix a Linux poll bug (a timeout of zero represents immediate timeout, not infinite as I though) and use allocation addresses for weak random data on the unsupported platforms.

[bors]

☔ The latest upstream changes (presumably #128134) made this pull request unmergeable. Please resolve the merge conflicts.

[joboet]

Closing in favour of #129201.