Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flake support #3573

Merged
merged 749 commits into from
Jul 16, 2020
Merged

Flake support #3573

merged 749 commits into from
Jul 16, 2020

Conversation

edolstra
Copy link
Member

@edolstra edolstra commented May 6, 2020

This PR adds experimental support for flakes. The flake file format is defined by RFC 49. To use it, you have to add

experimental-features = nix-command flakes

to nix.conf.

Other changes:

  • The nix command is flake-based now. For instance, nix build nixpkgs#hello builds the hello package from the nixpkgs flake. Non-flake-based operation is still supported, e.g. nix build -f /path/to/expr.nix. However, nix no longer uses NIX_PATH so it's no longer affected by channels.

  • New subcommand nix flake for working with flakes.

  • New subcommand nix profile for imperative package management (intended to replace nix-env). (This is not completely finished. Note that once a profile has been modified using nix profile, you can't use nix-env on it anymore.)

  • New subcommand nix run (previously named nix app) to run an application defined by an app or defaultApp output of a flake.

  • The nix command has an evaluation cache to speed up querying flake attributes. For instance, nix search uses this instead of the old JSON cache.

edolstra and others added 30 commits February 11, 2020 23:53
This allows all supported fetchers to be used, e.g.

  builtins.fetchTree {
    type = "github";
    owner = "NixOS";
    repo = "nix";
    rev = "d4df99a3349cf2228a8ee78dea320afef86eb3ba";
  }
Example usage:

  $ nix eval-hydra-jobs -f '<nixpkgs/pkgs/top-level/release.nix>' '' \
      --max-memory-size 2048 --workers 8
Looks like this got accidentally revived after 204291f.
Worktrees[1] are a feature of git which allow you to check out a ref in
a different directory.

While playing around with flakes I realized that git repositories in a
worktree checkout break when trying to build a flake:

```
$ git worktree add ../nixpkgs-flakes nixpkgs-flakes
$ cd ../nixpkgs-flakes
$ nix build .#hello
error: opening directory '/home/ma27/Projects/nixpkgs-flakes/.git/refs/heads': Not a directory
```

This issue has been fixed by determining with `git rev-parse --git-common-dir`
where the actual `.git` directory is.

Please note that this issue only exists on the `flakes` branch, fetching
worktree checkouts with Nix master seems to work fine.

[1] https://git-scm.com/docs/git-worktree
builtins.fetchGit: Fix build when fetching a git worktree
On second thought, let's move this back to Hydra.
Got this mixed up somewhere.
@mkenigs
Copy link
Contributor

mkenigs commented Jun 27, 2020

I've been thinking about usability, and I guess some of this might be more of my thoughts related to nix in general. But if I was trying to to convince someone to use nix, I would want a flake to look something more like this:

{
  description = "optional description";
  inputs.build = [ boost ];
  inputs.test = [ testframework ];
  outputs = { self, inputs }: {
    mkFlake {
      name = "name";
      build =
        ''
          make
        '';
      install =
        ''
          cp name $out/bin/
        '';
      run = "$out/bin/name";
      test =
        ''
          test
        '';
    }
  }
}

I don't know how things like nixpkgs, system, mkDerivation, and other outputs would be handled, so I guess this is more just my thoughts rather than something concrete. But if it was possible, I think something along these lines would be way easier to digest as a newcomer to nix. It's pretty easy to understand what's going on: you say what dependencies you need, nix does it's nix magic, and you get something you can build/run/develop/test anywhere with nix build/run/develop/test (flake check). I think it's hard to realize that's what nix can do when it's hidden behind complexity, but it's actually incredibly appealing once you see it.

edolstra and others added 22 commits June 29, 2020 14:14
…efinition

'nix run' will try to run $out/bin/<name>, where <name> is the
derivation name (excluding the version). This often works well:

  $ nix run nixpkgs#hello
  Hello, world!

  $ nix run nix -- --version
  nix (Nix) 2.4pre20200626_adf2fbb

  $ nix run patchelf -- --version
  patchelf 0.11.20200623.e61654b

  $ nix run nixpkgs#firefox -- --version
  Mozilla Firefox 77.0.1

  $ nix run nixpkgs#gimp -- --version
  GNU Image Manipulation Program version 2.10.14

though not always:

  $ nix run nixpkgs#git
  error: unable to execute '/nix/store/kp7wp760l4gryq9s36x481b2x4rfklcy-git-2.25.4/bin/git-minimal': No such file or directory
This allows you to refer to an input from another flake. For example,

  $ nix run --inputs-from /path/to/hydra nixpkgs#hello

runs 'hello' from the 'nixpkgs' inputs of the 'hydra' flake.

Fixes #3769.
Until now, the `gitlab`-fetcher determined the source's rev by checking
the latest commit of the given `ref` using the
`/repository/branches`-API.

This breaks however when trying to fetch a gitlab-repo by its tag:

```
$ nix repl
nix-repl> builtins.fetchTree gitlab:Ma27/nvim.nix/0.2.0
error: --- Error ------------------------------------------------------------------------------------- nix
unable to download 'https://gitlab.com/api/v4/projects/Ma27%2Fnvim.nix/repository/branches/0.2.0': HTTP error 404 ('')
```

When using the `/commits?ref_name`-endpoint[1] you can pass any kind of
valid ref to the `gitlab`-fetcher.

Please note that this fetches the only first 20 commits on a ref,
unfortunately there's currently no endpoint which only retrieves the
latest commit of any kind of `ref`.

[1] https://docs.gitlab.com/ee/api/commits.html#list-repository-commits
Fix gitlab-fetcher to obtain tags and branches
That is, the commands 'nix path-info nixpkgs#hello' and 'nix path-info
/nix/store/00ls0qi49qkqpqblmvz5s1ajl3gc63lr-hello-2.10.drv' now do the
same thing (i.e. build the derivation and operate on the output store
path, rather than the .drv path).
For instance, 'nix why-depends --use-derivation nixpkgs#hello
nixpkgs#glibc' shows why hello's .drv depends on glibc's .drv.
@edolstra
Copy link
Member Author

@mkenigs Agree completely but that's out of scope for the first iteration of flakes. In the future we might have a flake.toml containing a high-level description of a package, without any extraneous cruft like mkDerivation, function calls etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.