Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TCP support in the Nix daemon #5265

Draft
wants to merge 10 commits into
base: master
Choose a base branch
from
Draft

Conversation

edolstra
Copy link
Member

@edolstra edolstra commented Sep 17, 2021

This PR makes it possible to connect to a remote Nix daemon via TCP, and for the Nix daemon to listen on multiple sockets (including TCP) via systemd.

Lacking encryption/authentication, this is primarily useful to testing, e.g. when locally simulating high-latency SSH store connections via qdisc/netem.

Example usage:

# nix path-info --store tcp://example.org:1234 ...

@Ericson2314
Copy link
Member

Great!! I was hoping we'd do these things.

src/nix/daemon.md Outdated Show resolved Hide resolved
@edolstra edolstra changed the title Allow connecting to a Nix daemon via TCP TCP support in the Nix daemon Sep 21, 2021
edolstra and others added 10 commits September 23, 2021 11:21
Like UDSRemoteStore, but over a TCP connection. Example usage:

  $ nix path-info --store tcp://example.org:1234 ...
In conjunction with systemd socket activation, this allows the daemon
to listen to multiple sockets, e.g.

  [Socket]
  ListenStream=/nix/var/nix/daemon-socket/socket
  ListenStream=1234
Co-authored-by: John Ericson <git@JohnEricson.me>
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tweag-nix-dev-update-18/15300/1

@andir
Copy link
Member

andir commented Oct 8, 2021

I wonder if we really need that in Nix itself. if it is purely about testing locally wouldn't socat on the current unix socket work just fine?

@NickCao
Copy link
Member

NickCao commented Oct 8, 2021

But for building something like nixbuild.net, this can be extremely helpful.

@Ericson2314
Copy link
Member

To me this is good enough to merge I don't care about the daemon being able to open sockets of any sort because I basically believe one should always use socket activation.

I assume @edolstra is mainly waiting to get 2.4 our before returning to new features. That is also fine with me.

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/distributing-the-nix-store-with-cvmfs-nix/15706/2

@haslersn
Copy link

Lacking encryption/authentication, this is primarily useful to testing

Can you elaborate on this? What could an adversary with access to the TCP socket do, except for DoS (by consuming compute resources, bandwidth and disk space)? Is there a way for an adversary to compromise other users that use the same Nix daemon over TCP?

My potential use case: Sharing a long-living Nix daemon across short-living (and mutually distrusting) CI jobs.

@stale
Copy link

stale bot commented Jul 10, 2022

I marked this as stale due to inactivity. → More info

@stale stale bot added the stale label Jul 10, 2022
Ericson2314 added a commit to obsidiansystems/nix that referenced this pull request Feb 2, 2023
These settings are not needed for libstore at all, they are just used by
the nix daemon *command* for authorization on unix domain sockets. My
moving them to a new configuration struct just in that file, we avoid
them leaking anywhere else.

Also, it is good to break up the mammoth `Settings` struct in general.
Issue NixOS#5638 tracks this.

The message is not changed because I do not want to regress in
convenience to the user. Just saying "this connection is not trusted"
doesn't tell them out to fix the issue. The ideal thing to do would be
to somehow parameterize `processCommand` on how the error should be
displayed, so different sorts of connections can display different
information to the user based on how authentication is performed for the
connection in question. This, however, is a good bit more work, so it is
left for the future.

This came up with me thinking about the tcp:// store (NixOS#5265). The larger
project is not TCP *per se*, but the idea that it should be possible for
something else to manage access control to services like the Nix Daemon,
and those services simply trust or trust the incoming connection as they
are told. This is a more capability-oriented way of thinking about trust
than "every server implements its own auth separately" as we are used to today.

Its very great that libstore itself already implements just this model,
and so via this refactor I basically want to "enshrine" that so it
continues to be the case.
Ericson2314 added a commit to obsidiansystems/nix that referenced this pull request Feb 2, 2023
These settings are not needed for libstore at all, they are just used by
the nix daemon *command* for authorization on unix domain sockets. My
moving them to a new configuration struct just in that file, we avoid
them leaking anywhere else.

Also, it is good to break up the mammoth `Settings` struct in general.
Issue NixOS#5638 tracks this.

The message is not changed because I do not want to regress in
convenience to the user. Just saying "this connection is not trusted"
doesn't tell them out to fix the issue. The ideal thing to do would be
to somehow parameterize `processCommand` on how the error should be
displayed, so different sorts of connections can display different
information to the user based on how authentication is performed for the
connection in question. This, however, is a good bit more work, so it is
left for the future.

This came up with me thinking about the tcp:// store (NixOS#5265). The larger
project is not TCP *per se*, but the idea that it should be possible for
something else to manage access control to services like the Nix Daemon,
and those services simply trust or trust the incoming connection as they
are told. This is a more capability-oriented way of thinking about trust
than "every server implements its own auth separately" as we are used to today.

Its very great that libstore itself already implements just this model,
and so via this refactor I basically want to "enshrine" that so it
continues to be the case.
@Ericson2314
Copy link
Member

I have fixed the comments in this, but before I go too far proposing it I figured it would be good to start with something like #7739

@stale stale bot removed the stale label Feb 2, 2023
@Ericson2314
Copy link
Member

#6312 merges master into this.

Ericson2314 added a commit to obsidiansystems/nix that referenced this pull request Feb 23, 2023
We now support:

1. Multiple sockets, per the systemd socket activation spec

2. The sockets not having pid/uid/gid peer info because they might not
   be Unix domain sockets.

The changes are by @edolstra, taken from NixOS#5265. This is just that PR
*without* the TCP parts, which I gathered are the controversial parts.
Hopefully this remainder is not so controversial.
Ericson2314 added a commit to obsidiansystems/nix that referenced this pull request Mar 16, 2023
We now support:

1. Multiple sockets, per the systemd socket activation spec

2. The sockets not having pid/uid/gid peer info because they might not
   be Unix domain sockets.

The changes are by @edolstra, taken from NixOS#5265. This is just that PR
*without* the TCP parts, which I gathered are the controversial parts.
Hopefully this remainder is not so controversial.
Ericson2314 added a commit to obsidiansystems/nix that referenced this pull request Apr 7, 2023
We now support:

1. Multiple sockets, per the systemd socket activation spec

2. The sockets not having pid/uid/gid peer info because they might not
   be Unix domain sockets.

3. `--stdio` on the new `nix daemon`, not just the old `nix-daemon`.

In addition, `PeerInfo` is made more type safe with `std::optional`.

These changes are by @edolstra, taken from NixOS#5265. This is just that PR
*without* the TCP parts, which I gathered are the controversial parts.
Hopefully this remainder is not so controversial.
Ericson2314 added a commit to obsidiansystems/nix that referenced this pull request Apr 8, 2023
We now support:

1. Multiple sockets, per the systemd socket activation spec

2. The sockets not having pid/uid/gid peer info because they might not
   be Unix domain sockets.

3. `--stdio` on the new `nix daemon`, not just the old `nix-daemon`.

In addition, `PeerInfo` is made more type safe with `std::optional`.

These changes are by @edolstra, taken from NixOS#5265. This is just that PR
*without* the TCP parts, which I gathered are the controversial parts.
Hopefully this remainder is not so controversial.
Ericson2314 added a commit to obsidiansystems/nix that referenced this pull request May 10, 2023
We now support:

1. Multiple sockets, per the systemd socket activation spec

2. The sockets not having pid/uid/gid peer info because they might not
   be Unix domain sockets.

3. `--stdio` on the new `nix daemon`, not just the old `nix-daemon`.

In addition, `PeerInfo` is made more type safe with `std::optional`.

These changes are by @edolstra, taken from NixOS#5265. This is just that PR
*without* the TCP parts, which I gathered are the controversial parts.
Hopefully this remainder is not so controversial.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants