diff --git a/doc/manual/rl-next/shebang-relative.md b/doc/manual/rl-next/shebang-relative.md new file mode 100644 index 00000000000..c887a598a05 --- /dev/null +++ b/doc/manual/rl-next/shebang-relative.md @@ -0,0 +1,62 @@ +--- +synopsis: "`nix-shell` shebang uses relative path" +prs: +- 5088 +- 11058 +issues: +- 4232 +--- + + +Relative [path](@docroot@/language/values.md#type-path) literals in `nix-shell` shebang scripts' options are now resolved relative to the [script's location](@docroot@/glossary?highlight=base%20directory#gloss-base-directory). +Previously they were resolved relative to the current working directory. + +For example, consider the following script in `~/myproject/say-hi`: + +```shell +#!/usr/bin/env nix-shell +#!nix-shell --expr 'import ./shell.nix' +#!nix-shell --arg toolset './greeting-tools.nix' +#!nix-shell -i bash +hello +``` + +Older versions of `nix-shell` would resolve `shell.nix` relative to the current working directory; home in this example: + +```console +[hostname:~]$ ./myproject/say-hi +error: + … while calling the 'import' builtin + at «string»:1:2: + 1| (import ./shell.nix) + | ^ + + error: path '/home/user/shell.nix' does not exist +``` + +Since this release, `nix-shell` resolves `shell.nix` relative to the script's location, and `~/myproject/shell.nix` is used. + +```console +$ ./myproject/say-hi +Hello, world! +``` + +**Opt-out** + +This is technically a breaking change, so we have added an option so you can adapt independently of your Nix update. +The old behavior can be opted into by setting the option [`nix-shell-shebang-arguments-relative-to-script`](@docroot@/command-ref/conf-file.md#conf-nix-shell-shebang-arguments-relative-to-script) to `false`. +This option will be removed in a future release. + +**`nix` command shebang** + +The experimental [`nix` command shebang](@docroot@/command-ref/new-cli/nix.md?highlight=shebang#shebang-interpreter) already behaves in this script-relative manner. + +Example: + +```shell +#!/usr/bin/env nix +#!nix develop +#!nix --expr ``import ./shell.nix`` +#!nix -c bash +hello +``` diff --git a/doc/manual/src/language/syntax.md b/doc/manual/src/language/syntax.md index 238c502f979..b0779ea95b4 100644 --- a/doc/manual/src/language/syntax.md +++ b/doc/manual/src/language/syntax.md @@ -190,18 +190,13 @@ This section covers syntax and semantics of the Nix language. ### Path {#path-literal} - *Paths* are distinct from strings and can be expressed by path literals such as `./builder.sh`. - - Paths are suitable for referring to local files, and are often preferable over strings. - - Path values do not contain trailing slashes, `.` and `..`, as they are resolved when evaluating a path literal. - - Path literals are automatically resolved relative to their [base directory](@docroot@/glossary.md#gloss-base-directory). - - The files referred to by path values are automatically copied into the Nix store when used in a string interpolation or concatenation. - - Tooling can recognize path literals and provide additional features, such as autocompletion, refactoring automation and jump-to-file. + *Paths* can be expressed by path literals such as `./builder.sh`. A path literal must contain at least one slash to be recognised as such. For instance, `builder.sh` is not a path: it's parsed as an expression that selects the attribute `sh` from the variable `builder`. + Path literals are resolved relative to their [base directory](@docroot@/glossary.md#gloss-base-directory). Path literals may also refer to absolute paths by starting with a slash. > **Note** @@ -215,15 +210,6 @@ This section covers syntax and semantics of the Nix language. For example, `~/foo` would be equivalent to `/home/edolstra/foo` for a user whose home directory is `/home/edolstra`. Path literals that start with `~` are not allowed in [pure](@docroot@/command-ref/conf-file.md#conf-pure-eval) evaluation. - Paths can be used in [string interpolation] and string concatenation. - For instance, evaluating `"${./foo.txt}"` will cause `foo.txt` from the same directory to be copied into the Nix store and result in the string `"/nix/store/-foo.txt"`. - - Note that the Nix language assumes that all input files will remain _unchanged_ while evaluating a Nix expression. - For example, assume you used a file path in an interpolated string during a `nix repl` session. - Later in the same session, after having changed the file contents, evaluating the interpolated string with the file path again might not return a new [store path], since Nix might not re-read the file contents. Use `:r` to reset the repl as needed. - - [store path]: @docroot@/store/store-path.md - Path literals can also include [string interpolation], besides being [interpolated into other expressions]. [interpolated into other expressions]: ./string-interpolation.md#interpolated-expressions diff --git a/doc/manual/src/language/types.md b/doc/manual/src/language/types.md index c6cfb3c6909..229756e6be2 100644 --- a/doc/manual/src/language/types.md +++ b/doc/manual/src/language/types.md @@ -50,8 +50,37 @@ The function [`builtins.isString`](builtins.md#builtins-isString) can be used to ### Path {#type-path} - +A _path_ in the Nix language is an immutable, finite-length sequence of bytes starting with `/`, representing a POSIX-style, canonical file system path. +Path values are distinct from string values, even if they contain the same sequence of bytes. +Operations that produce paths will simplify the result as the standard C function [`realpath`] would, except that there is no symbolic link resolution. +[`realpath`]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/realpath.html + +Paths are suitable for referring to local files, and are often preferable over strings. +- Path values do not contain trailing or duplicate slashes, `.`, or `..`. +- Relative path literals are automatically resolved relative to their [base directory]. +- Tooling can recognize path literals and provide additional features, such as autocompletion, refactoring automation and jump-to-file. + +[base directory]: @docroot@/glossary.md#gloss-base-directory + +A file is not required to exist at a given path in order for that path value to be valid, but a path that is converted to a string with [string interpolation] or [string-and-path concatenation] must resolve to a readable file or directory which will be copied into the Nix store. +For instance, evaluating `"${./foo.txt}"` will cause `foo.txt` from the same directory to be copied into the Nix store and result in the string `"/nix/store/-foo.txt"`. +Operations such as [`import`] can also expect a path to resolve to a readable file or directory. + +[string interpolation]: string-interpolation.md#interpolated-expression +[string-and-path concatenation]: operators.md#string-and-path-concatenation +[`import`]: builtins.md#builtins-import + +> **Note** +> +> The Nix language assumes that all input files will remain _unchanged_ while evaluating a Nix expression. +> For example, assume you used a file path in an interpolated string during a `nix repl` session. +> Later in the same session, after having changed the file contents, evaluating the interpolated string with the file path again might not return a new [store path], since Nix might not re-read the file contents. +> Use `:r` to reset the repl as needed. + +[store path]: @docroot@/store/store-path.md + +Path values can be expressed as [path literals](syntax.md#path-literal). The function [`builtins.isPath`](builtins.md#builtins-isPath) can be used to determine if a value is a path. ### Null {#type-null} diff --git a/doc/manual/src/protocols/tarball-fetcher.md b/doc/manual/src/protocols/tarball-fetcher.md index 24ec7ae14aa..5cff05d66ce 100644 --- a/doc/manual/src/protocols/tarball-fetcher.md +++ b/doc/manual/src/protocols/tarball-fetcher.md @@ -41,4 +41,30 @@ Link: ///archive/.tar.gz +``` + +> **Example** +> +> +> ```nix +> # flake.nix +> { +> inputs = { +> foo.url = "https://gitea.example.org/some-person/some-flake/archive/main.tar.gz"; +> bar.url = "https://gitea.example.org/some-other-person/other-flake/archive/442793d9ec0584f6a6e82fa253850c8085bb150a.tar.gz"; +> qux = { +> url = "https://forgejo.example.org/another-person/some-non-flake-repo/archive/development.tar.gz"; +> flake = false; +> }; +> }; +> outputs = { foo, bar, qux }: { /* ... */ }; +> } +``` + [Nix Archive]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive diff --git a/flake.nix b/flake.nix index d83c2ecad36..51dbc309108 100644 --- a/flake.nix +++ b/flake.nix @@ -324,6 +324,7 @@ ++ pkgs.nixComponents.nix-external-api-docs.nativeBuildInputs ++ [ pkgs.buildPackages.cmake + pkgs.shellcheck modular.pre-commit.settings.package (pkgs.writeScriptBin "pre-commit-hooks-install" modular.pre-commit.settings.installationScript) diff --git a/maintainers/flake-module.nix b/maintainers/flake-module.nix index 561712badb2..67a426c23e7 100644 --- a/maintainers/flake-module.nix +++ b/maintainers/flake-module.nix @@ -495,7 +495,6 @@ excludes = [ # We haven't linted these files yet ''^config/install-sh$'' - ''^misc/systemv/nix-daemon$'' ''^misc/bash/completion\.sh$'' ''^misc/fish/completion\.fish$'' ''^misc/zsh/completion\.zsh$'' diff --git a/meson.build b/meson.build index e6bdc2eac30..1c46c5c2881 100644 --- a/meson.build +++ b/meson.build @@ -26,6 +26,7 @@ subproject('external-api-docs') subproject('libutil-c') subproject('libstore-c') subproject('libexpr-c') +subproject('libmain-c') # Language Bindings subproject('perl') diff --git a/misc/systemv/nix-daemon b/misc/systemv/nix-daemon index fea53716721..e8326f9472b 100755 --- a/misc/systemv/nix-daemon +++ b/misc/systemv/nix-daemon @@ -34,6 +34,7 @@ else fi # Source function library. +# shellcheck source=/dev/null . /etc/init.d/functions LOCKFILE=/var/lock/subsys/nix-daemon @@ -41,14 +42,20 @@ RUNDIR=/var/run/nix PIDFILE=${RUNDIR}/nix-daemon.pid RETVAL=0 -base=${0##*/} +# https://www.shellcheck.net/wiki/SC3004 +# Check if gettext exists +if ! type gettext > /dev/null 2>&1 +then + # If not, create a dummy function that returns the input verbatim + gettext() { printf '%s' "$1"; } +fi start() { mkdir -p ${RUNDIR} chown ${NIX_DAEMON_USER}:${NIX_DAEMON_USER} ${RUNDIR} - echo -n $"Starting nix daemon... " + printf '%s' "$(gettext 'Starting nix daemon... ')" daemonize -u $NIX_DAEMON_USER -p ${PIDFILE} $NIX_DAEMON_BIN $NIX_DAEMON_OPTS RETVAL=$? @@ -58,7 +65,7 @@ start() { } stop() { - echo -n $"Shutting down nix daemon: " + printf '%s' "$(gettext 'Shutting down nix daemon: ')" killproc -p ${PIDFILE} $NIX_DAEMON_BIN RETVAL=$? [ $RETVAL -eq 0 ] && rm -f ${LOCKFILE} ${PIDFILE} @@ -67,7 +74,7 @@ stop() { } reload() { - echo -n $"Reloading nix daemon... " + printf '%s' "$(gettext 'Reloading nix daemon... ')" killproc -p ${PIDFILE} $NIX_DAEMON_BIN -HUP RETVAL=$? echo @@ -105,7 +112,7 @@ case "$1" in fi ;; *) - echo $"Usage: $0 {start|stop|status|restart|condrestart}" + printf '%s' "$(gettext "Usage: $0 {start|stop|status|restart|condrestart}")" exit 2 ;; esac diff --git a/packaging/components.nix b/packaging/components.nix index 0e369a055cd..870e9ae615f 100644 --- a/packaging/components.nix +++ b/packaging/components.nix @@ -29,6 +29,7 @@ in nix-flake-tests = callPackage ../tests/unit/libflake/package.nix { }; nix-main = callPackage ../src/libmain/package.nix { }; + nix-main-c = callPackage ../src/libmain-c/package.nix { }; nix-cmd = callPackage ../src/libcmd/package.nix { }; diff --git a/packaging/hydra.nix b/packaging/hydra.nix index 4dfaf9bbfaa..dbe99247675 100644 --- a/packaging/hydra.nix +++ b/packaging/hydra.nix @@ -52,6 +52,7 @@ let "nix-flake" "nix-flake-tests" "nix-main" + "nix-main-c" "nix-cmd" "nix-ng" ]; diff --git a/src/build-remote/build-remote.cc b/src/build-remote/build-remote.cc index 600fc7ee2b7..1824e20024e 100644 --- a/src/build-remote/build-remote.cc +++ b/src/build-remote/build-remote.cc @@ -11,11 +11,12 @@ #include "machines.hh" #include "shared.hh" +#include "plugin.hh" #include "pathlocks.hh" #include "globals.hh" #include "serialise.hh" #include "build-result.hh" -#include "store-api.hh" +#include "store-open.hh" #include "strings.hh" #include "derivations.hh" #include "local-store.hh" diff --git a/src/libcmd/command.cc b/src/libcmd/command.cc index 67fef190920..349fb37d704 100644 --- a/src/libcmd/command.cc +++ b/src/libcmd/command.cc @@ -2,7 +2,7 @@ #include "command.hh" #include "markdown.hh" -#include "store-api.hh" +#include "store-open.hh" #include "local-fs-store.hh" #include "derivations.hh" #include "nixexpr.hh" diff --git a/src/libcmd/common-eval-args.cc b/src/libcmd/common-eval-args.cc index 470a25c4ec9..decadd751cc 100644 --- a/src/libcmd/common-eval-args.cc +++ b/src/libcmd/common-eval-args.cc @@ -214,7 +214,7 @@ Bindings * MixEvalArgs::getAutoArgs(EvalState & state) auto v = state.allocValue(); std::visit(overloaded { [&](const AutoArgExpr & arg) { - state.mkThunk_(*v, state.parseExprFromString(arg.expr, state.rootPath("."))); + state.mkThunk_(*v, state.parseExprFromString(arg.expr, compatibilitySettings.nixShellShebangArgumentsRelativeToScript ? state.rootPath(absPath(getCommandBaseDir())) : state.rootPath("."))); }, [&](const AutoArgString & arg) { v->mkString(arg.s); diff --git a/src/libcmd/compatibility-settings.hh b/src/libcmd/compatibility-settings.hh index 5dc0eaf2b38..a129a957a64 100644 --- a/src/libcmd/compatibility-settings.hh +++ b/src/libcmd/compatibility-settings.hh @@ -7,12 +7,29 @@ struct CompatibilitySettings : public Config CompatibilitySettings() = default; + // Added in Nix 2.24, July 2024. Setting nixShellAlwaysLooksForShellNix{this, true, "nix-shell-always-looks-for-shell-nix", R"( Before Nix 2.24, [`nix-shell`](@docroot@/command-ref/nix-shell.md) would only look at `shell.nix` if it was in the working directory - when no file was specified. Since Nix 2.24, `nix-shell` always looks for a `shell.nix`, whether that's in the working directory, or in a directory that was passed as an argument. - You may set this to `false` to revert to the Nix 2.3 behavior. + You may set this to `false` to temporarily revert to the behavior of Nix 2.23 and older. + + Using this setting is not recommended. + It will be deprecated and removed. + )"}; + + // Added in Nix 2.24, July 2024. + Setting nixShellShebangArgumentsRelativeToScript{ + this, true, "nix-shell-shebang-arguments-relative-to-script", R"( + Before Nix 2.24, relative file path expressions in arguments in a `nix-shell` shebang were resolved relative to the working directory. + + Since Nix 2.24, `nix-shell` resolves these paths in a manner that is relative to the [base directory](@docroot@/glossary.md#gloss-base-directory), defined as the script's directory. + + You may set this to `false` to temporarily revert to the behavior of Nix 2.23 and older. + + Using this setting is not recommended. + It will be deprecated and removed. )"}; }; diff --git a/src/libexpr/eval-settings.cc b/src/libexpr/eval-settings.cc index e2151aa7fc1..eb5761638a7 100644 --- a/src/libexpr/eval-settings.cc +++ b/src/libexpr/eval-settings.cc @@ -56,7 +56,7 @@ EvalSettings::EvalSettings(bool & readOnlyMode, EvalSettings::LookupPathHooks lo builtinsAbortOnWarn = true; } -Strings EvalSettings::getDefaultNixPath() const +Strings EvalSettings::getDefaultNixPath() { Strings res; auto add = [&](const Path & p, const std::string & s = std::string()) { @@ -69,11 +69,9 @@ Strings EvalSettings::getDefaultNixPath() const } }; - if (!restrictEval && !pureEval) { - add(getNixDefExpr() + "/channels"); - add(rootChannelsDir() + "/nixpkgs", "nixpkgs"); - add(rootChannelsDir()); - } + add(getNixDefExpr() + "/channels"); + add(rootChannelsDir() + "/nixpkgs", "nixpkgs"); + add(rootChannelsDir()); return res; } diff --git a/src/libexpr/eval-settings.hh b/src/libexpr/eval-settings.hh index b915ba530a0..89a42caba5d 100644 --- a/src/libexpr/eval-settings.hh +++ b/src/libexpr/eval-settings.hh @@ -43,7 +43,7 @@ struct EvalSettings : Config bool & readOnlyMode; - Strings getDefaultNixPath() const; + static Strings getDefaultNixPath(); static bool isPseudoUrl(std::string_view s); diff --git a/src/libexpr/eval.hh b/src/libexpr/eval.hh index d376046ae30..f09e6223a77 100644 --- a/src/libexpr/eval.hh +++ b/src/libexpr/eval.hh @@ -130,7 +130,7 @@ struct Constant typedef std::map ValMap; #endif -typedef std::map DocCommentMap; +typedef std::unordered_map DocCommentMap; struct Env { @@ -335,7 +335,7 @@ private: * Associate source positions of certain AST nodes with their preceding doc comment, if they have one. * Grouped by file. */ - std::map positionToDocComment; + std::unordered_map positionToDocComment; LookupPath lookupPath; diff --git a/src/libexpr/parser-state.hh b/src/libexpr/parser-state.hh index 983a17a2e98..5a7bcb7175d 100644 --- a/src/libexpr/parser-state.hh +++ b/src/libexpr/parser-state.hh @@ -64,7 +64,7 @@ struct LexerState /** * @brief Maps some positions to a DocComment, where the comment is relevant to the location. */ - std::map & positionToDocComment; + std::unordered_map & positionToDocComment; PosTable & positions; PosTable::Origin origin; diff --git a/src/libexpr/parser.y b/src/libexpr/parser.y index 452d265bcbf..8ea176b2461 100644 --- a/src/libexpr/parser.y +++ b/src/libexpr/parser.y @@ -48,7 +48,7 @@ namespace nix { -typedef std::map DocCommentMap; +typedef std::unordered_map DocCommentMap; Expr * parseExprFromBuf( char * text, diff --git a/src/libexpr/pos-idx.hh b/src/libexpr/pos-idx.hh index e1349156022..2faa6b7fe4f 100644 --- a/src/libexpr/pos-idx.hh +++ b/src/libexpr/pos-idx.hh @@ -1,6 +1,7 @@ #pragma once #include +#include namespace nix { @@ -8,6 +9,7 @@ class PosIdx { friend struct LazyPosAcessors; friend class PosTable; + friend class std::hash; private: uint32_t id; @@ -37,8 +39,26 @@ public: { return id == other.id; } + + size_t hash() const noexcept + { + return std::hash{}(id); + } }; inline PosIdx noPos = {}; } + +namespace std { + +template<> +struct hash +{ + std::size_t operator()(nix::PosIdx pos) const noexcept + { + return pos.hash(); + } +}; + +} // namespace std diff --git a/src/libexpr/primops.cc b/src/libexpr/primops.cc index 127823d4944..5a373a43bda 100644 --- a/src/libexpr/primops.cc +++ b/src/libexpr/primops.cc @@ -719,7 +719,7 @@ static RegisterPrimOp primop_genericClosure(PrimOp { .doc = R"( `builtins.genericClosure` iteratively computes the transitive closure over an arbitrary relation defined by a function. - It takes *attrset* with two attributes named `startSet` and `operator`, and returns a list of attrbute sets: + It takes *attrset* with two attributes named `startSet` and `operator`, and returns a list of attribute sets: - `startSet`: The initial list of attribute sets. diff --git a/src/libexpr/primops/fetchClosure.cc b/src/libexpr/primops/fetchClosure.cc index fc5bb31454f..7f59cac2f70 100644 --- a/src/libexpr/primops/fetchClosure.cc +++ b/src/libexpr/primops/fetchClosure.cc @@ -1,5 +1,5 @@ #include "primops.hh" -#include "store-api.hh" +#include "store-open.hh" #include "realisation.hh" #include "make-content-addressed.hh" #include "url.hh" diff --git a/src/libexpr/print.cc b/src/libexpr/print.cc index 2f377e5886e..bc17d6bfe26 100644 --- a/src/libexpr/print.cc +++ b/src/libexpr/print.cc @@ -299,6 +299,9 @@ class Printer output << ANSI_NORMAL; } + /** + * @note This may force items. + */ bool shouldPrettyPrintAttrs(AttrVec & v) { if (!options.shouldPrettyPrint() || v.empty()) { @@ -315,6 +318,9 @@ class Printer return true; } + // It is ok to force the item(s) here, because they will be printed anyway. + state.forceValue(*item, item->determinePos(noPos)); + // Pretty-print single-item attrsets only if they contain nested // structures. auto itemType = item->type(); @@ -371,6 +377,9 @@ class Printer } } + /** + * @note This may force items. + */ bool shouldPrettyPrintList(std::span list) { if (!options.shouldPrettyPrint() || list.empty()) { @@ -387,6 +396,9 @@ class Printer return true; } + // It is ok to force the item(s) here, because they will be printed anyway. + state.forceValue(*item, item->determinePos(noPos)); + // Pretty-print single-item lists only if they contain nested // structures. auto itemType = item->type(); diff --git a/src/libmain-c/.version b/src/libmain-c/.version new file mode 120000 index 00000000000..b7badcd0cc8 --- /dev/null +++ b/src/libmain-c/.version @@ -0,0 +1 @@ +../../.version \ No newline at end of file diff --git a/src/libmain-c/build-utils-meson b/src/libmain-c/build-utils-meson new file mode 120000 index 00000000000..5fff21bab55 --- /dev/null +++ b/src/libmain-c/build-utils-meson @@ -0,0 +1 @@ +../../build-utils-meson \ No newline at end of file diff --git a/src/libmain-c/meson.build b/src/libmain-c/meson.build new file mode 100644 index 00000000000..1d6b2f959ff --- /dev/null +++ b/src/libmain-c/meson.build @@ -0,0 +1,84 @@ +project('nix-main-c', 'cpp', + version : files('.version'), + default_options : [ + 'cpp_std=c++2a', + # TODO(Qyriad): increase the warning level + 'warning_level=1', + 'debug=true', + 'optimization=2', + 'errorlogs=true', # Please print logs for tests that fail + ], + meson_version : '>= 1.1', + license : 'LGPL-2.1-or-later', +) + +cxx = meson.get_compiler('cpp') + +subdir('build-utils-meson/deps-lists') + +configdata = configuration_data() + +deps_private_maybe_subproject = [ + dependency('nix-util'), + dependency('nix-store'), + dependency('nix-main'), +] +deps_public_maybe_subproject = [ + dependency('nix-util-c'), + dependency('nix-store-c'), +] +subdir('build-utils-meson/subprojects') + +# TODO rename, because it will conflict with downstream projects +configdata.set_quoted('PACKAGE_VERSION', meson.project_version()) + +config_h = configure_file( + configuration : configdata, + output : 'config-main.h', +) + +add_project_arguments( + # TODO(Qyriad): Yes this is how the autoconf+Make system did it. + # It would be nice for our headers to be idempotent instead. + + # From C++ libraries, only for internals + '-include', 'config-util.hh', + '-include', 'config-store.hh', + '-include', 'config-main.hh', + + # From C libraries, for our public, installed headers too + '-include', 'config-util.h', + '-include', 'config-store.h', + '-include', 'config-main.h', + language : 'cpp', +) + +subdir('build-utils-meson/diagnostics') + +sources = files( + 'nix_api_main.cc', +) + +include_dirs = [include_directories('.')] + +headers = [config_h] + files( + 'nix_api_main.h', +) + +subdir('build-utils-meson/export-all-symbols') + +this_library = library( + 'nixmainc', + sources, + dependencies : deps_public + deps_private + deps_other, + include_directories : include_dirs, + link_args: linker_export_flags, + prelink : true, # For C++ static initializers + install : true, +) + +install_headers(headers, subdir : 'nix', preserve_path : true) + +libraries_private = [] + +subdir('build-utils-meson/export') diff --git a/src/libmain-c/nix_api_main.cc b/src/libmain-c/nix_api_main.cc new file mode 100644 index 00000000000..692d53f47e0 --- /dev/null +++ b/src/libmain-c/nix_api_main.cc @@ -0,0 +1,16 @@ +#include "nix_api_store.h" +#include "nix_api_store_internal.h" +#include "nix_api_util.h" +#include "nix_api_util_internal.h" + +#include "plugin.hh" + +nix_err nix_init_plugins(nix_c_context * context) +{ + if (context) + context->last_err_code = NIX_OK; + try { + nix::initPlugins(); + } + NIXC_CATCH_ERRS +} diff --git a/src/libmain-c/nix_api_main.h b/src/libmain-c/nix_api_main.h new file mode 100644 index 00000000000..3957b992fd3 --- /dev/null +++ b/src/libmain-c/nix_api_main.h @@ -0,0 +1,40 @@ +#ifndef NIX_API_MAIN_H +#define NIX_API_MAIN_H +/** + * @defgroup libmain libmain + * @brief C bindings for nix libmain + * + * libmain has misc utilities for CLI commands + * @{ + */ +/** @file + * @brief Main entry for the libmain C bindings + */ + +#include "nix_api_util.h" +#include + +#ifdef __cplusplus +extern "C" { +#endif +// cffi start + +/** + * @brief Loads the plugins specified in Nix's plugin-files setting. + * + * Call this once, after calling your desired init functions and setting + * relevant settings. + * + * @param[out] context Optional, stores error information + * @return NIX_OK if the initialization was successful, an error code otherwise. + */ +nix_err nix_init_plugins(nix_c_context * context); + +// cffi end +#ifdef __cplusplus +} +#endif +/** + * @} + */ +#endif // NIX_API_MAIN_H diff --git a/src/libmain-c/package.nix b/src/libmain-c/package.nix new file mode 100644 index 00000000000..478e34a856a --- /dev/null +++ b/src/libmain-c/package.nix @@ -0,0 +1,83 @@ +{ lib +, stdenv +, mkMesonDerivation +, releaseTools + +, meson +, ninja +, pkg-config + +, nix-util-c +, nix-store +, nix-store-c +, nix-main + +# Configuration Options + +, version +}: + +let + inherit (lib) fileset; +in + +mkMesonDerivation (finalAttrs: { + pname = "nix-main-c"; + inherit version; + + workDir = ./.; + fileset = fileset.unions [ + ../../build-utils-meson + ./build-utils-meson + ../../.version + ./.version + ./meson.build + # ./meson.options + (fileset.fileFilter (file: file.hasExt "cc") ./.) + (fileset.fileFilter (file: file.hasExt "hh") ./.) + (fileset.fileFilter (file: file.hasExt "h") ./.) + ]; + + outputs = [ "out" "dev" ]; + + nativeBuildInputs = [ + meson + ninja + pkg-config + ]; + + propagatedBuildInputs = [ + nix-util-c + nix-store + nix-store-c + nix-main + ]; + + preConfigure = + # "Inline" .version so it's not a symlink, and includes the suffix. + # Do the meson utils, without modification. + '' + chmod u+w ./.version + echo ${version} > ../../.version + ''; + + mesonFlags = [ + ]; + + env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { + LDFLAGS = "-fuse-ld=gold"; + }; + + enableParallelBuilding = true; + + separateDebugInfo = !stdenv.hostPlatform.isStatic; + + strictDeps = true; + + hardeningDisable = lib.optional stdenv.hostPlatform.isStatic "pie"; + + meta = { + platforms = lib.platforms.unix ++ lib.platforms.windows; + }; + +}) diff --git a/src/libmain/args/root.hh b/src/libmain/args/root.hh index 5c55c37a55d..34a43b53835 100644 --- a/src/libmain/args/root.hh +++ b/src/libmain/args/root.hh @@ -29,6 +29,7 @@ struct Completions final : AddCompletions */ class RootArgs : virtual public Args { +protected: /** * @brief The command's "working directory", but only set when top level. * diff --git a/src/libmain/common-args.cc b/src/libmain/common-args.cc index a94845ab8f3..768b2177c94 100644 --- a/src/libmain/common-args.cc +++ b/src/libmain/common-args.cc @@ -5,6 +5,7 @@ #include "logging.hh" #include "loggers.hh" #include "util.hh" +#include "plugin.hh" namespace nix { diff --git a/src/libmain/meson.build b/src/libmain/meson.build index 859ce22f8ba..fe6133596a9 100644 --- a/src/libmain/meson.build +++ b/src/libmain/meson.build @@ -64,6 +64,7 @@ subdir('build-utils-meson/diagnostics') sources = files( 'common-args.cc', 'loggers.cc', + 'plugin.cc', 'progress-bar.cc', 'shared.cc', ) @@ -79,6 +80,7 @@ include_dirs = [include_directories('.')] headers = [config_h] + files( 'common-args.hh', 'loggers.hh', + 'plugin.hh', 'progress-bar.hh', 'shared.hh', ) diff --git a/src/libmain/plugin.cc b/src/libmain/plugin.cc new file mode 100644 index 00000000000..52b5b60e415 --- /dev/null +++ b/src/libmain/plugin.cc @@ -0,0 +1,117 @@ +#ifndef _WIN32 +# include +#endif + +#include "config-global.hh" +#include "signals.hh" + +namespace nix { + +struct PluginFilesSetting : public BaseSetting +{ + bool pluginsLoaded = false; + + PluginFilesSetting( + Config * options, + const Paths & def, + const std::string & name, + const std::string & description, + const std::set & aliases = {}) + : BaseSetting(def, true, name, description, aliases) + { + options->addSetting(this); + } + + Paths parse(const std::string & str) const override; +}; + +Paths PluginFilesSetting::parse(const std::string & str) const +{ + if (pluginsLoaded) + throw UsageError( + "plugin-files set after plugins were loaded, you may need to move the flag before the subcommand"); + return BaseSetting::parse(str); +} + +struct PluginSettings : Config +{ + PluginFilesSetting pluginFiles{ + this, + {}, + "plugin-files", + R"( + A list of plugin files to be loaded by Nix. Each of these files will + be dlopened by Nix. If they contain the symbol `nix_plugin_entry()`, + this symbol will be called. Alternatively, they can affect execution + through static initialization. In particular, these plugins may construct + static instances of RegisterPrimOp to add new primops or constants to the + expression language, RegisterStoreImplementation to add new store + implementations, RegisterCommand to add new subcommands to the `nix` + command, and RegisterSetting to add new nix config settings. See the + constructors for those types for more details. + + Warning! These APIs are inherently unstable and may change from + release to release. + + Since these files are loaded into the same address space as Nix + itself, they must be DSOs compatible with the instance of Nix + running at the time (i.e. compiled against the same headers, not + linked to any incompatible libraries). They should not be linked to + any Nix libs directly, as those will be available already at load + time. + + If an entry in the list is a directory, all files in the directory + are loaded as plugins (non-recursively). + )"}; +}; + +static PluginSettings pluginSettings; + +static GlobalConfig::Register rPluginSettings(&pluginSettings); + +void initPlugins() +{ + assert(!pluginSettings.pluginFiles.pluginsLoaded); + for (const auto & pluginFile : pluginSettings.pluginFiles.get()) { + std::vector pluginFiles; + try { + auto ents = std::filesystem::directory_iterator{pluginFile}; + for (const auto & ent : ents) { + checkInterrupt(); + pluginFiles.emplace_back(ent.path()); + } + } catch (std::filesystem::filesystem_error & e) { + if (e.code() != std::errc::not_a_directory) + throw; + pluginFiles.emplace_back(pluginFile); + } + for (const auto & file : pluginFiles) { + checkInterrupt(); + /* handle is purposefully leaked as there may be state in the + DSO needed by the action of the plugin. */ +#ifndef _WIN32 // TODO implement via DLL loading on Windows + void * handle = dlopen(file.c_str(), RTLD_LAZY | RTLD_LOCAL); + if (!handle) + throw Error("could not dynamically open plugin file '%s': %s", file, dlerror()); + + /* Older plugins use a statically initialized object to run their code. + Newer plugins can also export nix_plugin_entry() */ + void (*nix_plugin_entry)() = (void (*)()) dlsym(handle, "nix_plugin_entry"); + if (nix_plugin_entry) + nix_plugin_entry(); +#else + throw Error("could not dynamically open plugin file '%s'", file); +#endif + } + } + + /* Since plugins can add settings, try to re-apply previously + unknown settings. */ + globalConfig.reapplyUnknownSettings(); + globalConfig.warnUnknownSettings(); + + /* Tell the user if they try to set plugin-files after we've already loaded */ + pluginSettings.pluginFiles.pluginsLoaded = true; +} + +} diff --git a/src/libmain/plugin.hh b/src/libmain/plugin.hh new file mode 100644 index 00000000000..4221c1b1713 --- /dev/null +++ b/src/libmain/plugin.hh @@ -0,0 +1,12 @@ +#pragma once +///@file + +namespace nix { + +/** + * This should be called after settings are initialized, but before + * anything else + */ +void initPlugins(); + +} diff --git a/src/libstore-c/nix_api_store.cc b/src/libstore-c/nix_api_store.cc index 4fe25c7d4ee..4961937cf12 100644 --- a/src/libstore-c/nix_api_store.cc +++ b/src/libstore-c/nix_api_store.cc @@ -5,6 +5,7 @@ #include "path.hh" #include "store-api.hh" +#include "store-open.hh" #include "build-result.hh" #include "globals.hh" @@ -29,16 +30,6 @@ nix_err nix_libstore_init_no_load_config(nix_c_context * context) NIXC_CATCH_ERRS } -nix_err nix_init_plugins(nix_c_context * context) -{ - if (context) - context->last_err_code = NIX_OK; - try { - nix::initPlugins(); - } - NIXC_CATCH_ERRS -} - Store * nix_store_open(nix_c_context * context, const char * uri, const char *** params) { if (context) @@ -52,7 +43,7 @@ Store * nix_store_open(nix_c_context * context, const char * uri, const char *** if (!params) return new Store{nix::openStore(uri_str)}; - nix::Store::Params params_map; + nix::StoreReference::Params params_map; for (size_t i = 0; params[i] != nullptr; i++) { params_map[params[i][0]] = params[i][1]; } diff --git a/src/libstore-c/nix_api_store.h b/src/libstore-c/nix_api_store.h index d3cb8fab84e..4b213445778 100644 --- a/src/libstore-c/nix_api_store.h +++ b/src/libstore-c/nix_api_store.h @@ -42,17 +42,6 @@ nix_err nix_libstore_init(nix_c_context * context); */ nix_err nix_libstore_init_no_load_config(nix_c_context * context); -/** - * @brief Loads the plugins specified in Nix's plugin-files setting. - * - * Call this once, after calling your desired init functions and setting - * relevant settings. - * - * @param[out] context Optional, stores error information - * @return NIX_OK if the initialization was successful, an error code otherwise. - */ -nix_err nix_init_plugins(nix_c_context * context); - /** * @brief Open a nix store. * diff --git a/src/libstore/binary-cache-store.hh b/src/libstore/binary-cache-store.hh index 695bc925277..234c39caf3a 100644 --- a/src/libstore/binary-cache-store.hh +++ b/src/libstore/binary-cache-store.hh @@ -96,7 +96,11 @@ public: public: - virtual void init() override; + /** + * Perform any necessary effectful operation to make the store up and + * running + */ + virtual void init(); private: diff --git a/src/libstore/build/derivation-goal.cc b/src/libstore/build/derivation-goal.cc index 010f905d640..b809e3ffe3f 100644 --- a/src/libstore/build/derivation-goal.cc +++ b/src/libstore/build/derivation-goal.cc @@ -1569,7 +1569,7 @@ void DerivationGoal::waiteeDone(GoalPtr waitee, ExitCode result) { Goal::waiteeDone(waitee, result); - if (!useDerivation) return; + if (!useDerivation || !drv) return; auto & fullDrv = *dynamic_cast(drv.get()); auto * dg = dynamic_cast(&*waitee); diff --git a/src/libstore/build/drv-output-substitution-goal.cc b/src/libstore/build/drv-output-substitution-goal.cc index 02284d93c1a..4488efdcb59 100644 --- a/src/libstore/build/drv-output-substitution-goal.cc +++ b/src/libstore/build/drv-output-substitution-goal.cc @@ -3,6 +3,7 @@ #include "worker.hh" #include "substitution-goal.hh" #include "callback.hh" +#include "store-open.hh" namespace nix { diff --git a/src/libstore/config-parse-impl.hh b/src/libstore/config-parse-impl.hh new file mode 100644 index 00000000000..ebb65305ae3 --- /dev/null +++ b/src/libstore/config-parse-impl.hh @@ -0,0 +1,24 @@ +#pragma once +//@file + +#include + +#include "config-parse.hh" +#include "util.hh" + +namespace nix::config { + +template +std::optional> SettingInfo::parseConfig(const nlohmann::json::object_t & map) const +{ + auto * p = get(map, name); + return p ? (JustValue{.value = p->get()}) : (std::optional>{}); +} + +/** + * Look up the setting's name in a map, falling back on the default if + * it does not exist. + */ +#define CONFIG_ROW(FIELD) .FIELD = descriptions.FIELD.parseConfig(params).value_or(defaults.FIELD) + +} diff --git a/src/libstore/config-parse.hh b/src/libstore/config-parse.hh index 52b0ee7772d..5d17f9f8d6a 100644 --- a/src/libstore/config-parse.hh +++ b/src/libstore/config-parse.hh @@ -1,13 +1,21 @@ #pragma once +//@file -/** - * Look up the setting's name in a map, falling back on the default if - * it does not exist. - */ -#define CONFIG_ROW(FIELD) \ - .FIELD = { \ - .value = ({ \ - auto p = get(params, descriptions.FIELD.name); \ - p ? *p : defaults.FIELD.value; \ - }), \ - } +#include + +#include "config-abstract.hh" +#include "util.hh" + +namespace nix::config { + +template +struct SettingInfo +{ + std::string name; + std::string description; + bool documentDefault = true; + + std::optional> parseConfig(const nlohmann::json::object_t & map) const; +}; + +} diff --git a/src/libstore/daemon.cc b/src/libstore/daemon.cc index 40163a6214d..5c5080f8aa9 100644 --- a/src/libstore/daemon.cc +++ b/src/libstore/daemon.cc @@ -246,10 +246,9 @@ struct ClientSettings // the daemon, as that could cause some pretty weird stuff if (parseFeatures(tokenizeString(value)) != experimentalFeatureSettings.experimentalFeatures.get()) debug("Ignoring the client-specified experimental features"); - } else if (name == settings.pluginFiles.name) { - if (tokenizeString(value) != settings.pluginFiles.get()) - warn("Ignoring the client-specified plugin-files.\n" - "The client specifying plugins to the daemon never made sense, and was removed in Nix >=2.14."); + } else if (name == "plugin-files") { + warn("Ignoring the client-specified plugin-files.\n" + "The client specifying plugins to the daemon never made sense, and was removed in Nix >=2.14."); } else if (trusted || name == settings.buildTimeout.name diff --git a/src/libstore/dummy-store.cc b/src/libstore/dummy-store.cc index c1e871e9384..c6dd71ba669 100644 --- a/src/libstore/dummy-store.cc +++ b/src/libstore/dummy-store.cc @@ -1,4 +1,5 @@ #include "store-api.hh" +#include "store-registration.hh" #include "callback.hh" namespace nix { @@ -6,8 +7,17 @@ namespace nix { struct DummyStoreConfig : virtual StoreConfig { using StoreConfig::StoreConfig; - DummyStoreConfig(std::string_view scheme, std::string_view authority, const Params & params) - : StoreConfig(params) + struct Descriptions : virtual Store::Config::Descriptions + { + Descriptions() + : StoreConfig::Descriptions{Store::Config::descriptions} + {} + }; + + static const Descriptions descriptions; + + DummyStoreConfig(std::string_view scheme, std::string_view authority, const StoreReference::Params & params) + : StoreConfig{params} { if (!authority.empty()) throw UsageError("`%s` store URIs must not contain an authority part %s", scheme, authority); @@ -25,18 +35,22 @@ struct DummyStoreConfig : virtual StoreConfig { static std::set uriSchemes() { return {"dummy"}; } + + std::shared_ptr openStore() override; }; + +const DummyStoreConfig::Descriptions DummyStoreConfig::descriptions{}; + + struct DummyStore : public virtual DummyStoreConfig, public virtual Store { - DummyStore(std::string_view scheme, std::string_view authority, const Params & params) - : StoreConfig(params) - , DummyStoreConfig(scheme, authority, params) - , Store(params) - { } + using Config = DummyStoreConfig; - DummyStore(const Params & params) - : DummyStore("dummy", "", params) + DummyStore(const Config & config) + : StoreConfig(config) + , DummyStoreConfig(config) + , Store{static_cast(*this)} { } std::string getUri() override @@ -86,6 +100,11 @@ struct DummyStore : public virtual DummyStoreConfig, public virtual Store { unsupported("getFSAccessor"); } }; -static RegisterStoreImplementation regDummyStore; +std::shared_ptr DummyStore::Config::openStore() +{ + return std::make_shared(*this); +} + +static RegisterStoreImplementation regDummyStore; } diff --git a/src/libstore/globals.cc b/src/libstore/globals.cc index 4eabf6054dc..fa4c0ba7fad 100644 --- a/src/libstore/globals.cc +++ b/src/libstore/globals.cc @@ -15,7 +15,6 @@ #include #ifndef _WIN32 -# include # include #endif @@ -335,60 +334,6 @@ unsigned int MaxBuildJobsSetting::parse(const std::string & str) const } -Paths PluginFilesSetting::parse(const std::string & str) const -{ - if (pluginsLoaded) - throw UsageError("plugin-files set after plugins were loaded, you may need to move the flag before the subcommand"); - return BaseSetting::parse(str); -} - - -void initPlugins() -{ - assert(!settings.pluginFiles.pluginsLoaded); - for (const auto & pluginFile : settings.pluginFiles.get()) { - std::vector pluginFiles; - try { - auto ents = std::filesystem::directory_iterator{pluginFile}; - for (const auto & ent : ents) { - checkInterrupt(); - pluginFiles.emplace_back(ent.path()); - } - } catch (std::filesystem::filesystem_error & e) { - if (e.code() != std::errc::not_a_directory) - throw; - pluginFiles.emplace_back(pluginFile); - } - for (const auto & file : pluginFiles) { - checkInterrupt(); - /* handle is purposefully leaked as there may be state in the - DSO needed by the action of the plugin. */ -#ifndef _WIN32 // TODO implement via DLL loading on Windows - void *handle = - dlopen(file.c_str(), RTLD_LAZY | RTLD_LOCAL); - if (!handle) - throw Error("could not dynamically open plugin file '%s': %s", file, dlerror()); - - /* Older plugins use a statically initialized object to run their code. - Newer plugins can also export nix_plugin_entry() */ - void (*nix_plugin_entry)() = (void (*)())dlsym(handle, "nix_plugin_entry"); - if (nix_plugin_entry) - nix_plugin_entry(); -#else - throw Error("could not dynamically open plugin file '%s'", file); -#endif - } - } - - /* Since plugins can add settings, try to re-apply previously - unknown settings. */ - globalConfig.reapplyUnknownSettings(); - globalConfig.warnUnknownSettings(); - - /* Tell the user if they try to set plugin-files after we've already loaded */ - settings.pluginFiles.pluginsLoaded = true; -} - static void preloadNSS() { /* builtin:fetchurl can trigger a DNS lookup, which with glibc can trigger a dynamic library load of diff --git a/src/libstore/globals.hh b/src/libstore/globals.hh index dfe25f31726..30d7537bd52 100644 --- a/src/libstore/globals.hh +++ b/src/libstore/globals.hh @@ -31,23 +31,6 @@ struct MaxBuildJobsSetting : public BaseSetting unsigned int parse(const std::string & str) const override; }; -struct PluginFilesSetting : public BaseSetting -{ - bool pluginsLoaded = false; - - PluginFilesSetting(Config * options, - const Paths & def, - const std::string & name, - const std::string & description, - const std::set & aliases = {}) - : BaseSetting(def, true, name, description, aliases) - { - options->addSetting(this); - } - - Paths parse(const std::string & str) const override; -}; - const uint32_t maxIdsPerBuild = #if __linux__ 1 << 16 @@ -1158,33 +1141,6 @@ public: Setting minFreeCheckInterval{this, 5, "min-free-check-interval", "Number of seconds between checking free disk space."}; - PluginFilesSetting pluginFiles{ - this, {}, "plugin-files", - R"( - A list of plugin files to be loaded by Nix. Each of these files will - be dlopened by Nix. If they contain the symbol `nix_plugin_entry()`, - this symbol will be called. Alternatively, they can affect execution - through static initialization. In particular, these plugins may construct - static instances of RegisterPrimOp to add new primops or constants to the - expression language, RegisterStoreImplementation to add new store - implementations, RegisterCommand to add new subcommands to the `nix` - command, and RegisterSetting to add new nix config settings. See the - constructors for those types for more details. - - Warning! These APIs are inherently unstable and may change from - release to release. - - Since these files are loaded into the same address space as Nix - itself, they must be DSOs compatible with the instance of Nix - running at the time (i.e. compiled against the same headers, not - linked to any incompatible libraries). They should not be linked to - any Nix libs directly, as those will be available already at load - time. - - If an entry in the list is a directory, all files in the directory - are loaded as plugins (non-recursively). - )"}; - Setting narBufferSize{this, 32 * 1024 * 1024, "nar-buffer-size", "Maximum size of NARs before spilling them to disk."}; @@ -1278,12 +1234,6 @@ public: // FIXME: don't use a global variable. extern Settings settings; -/** - * This should be called after settings are initialized, but before - * anything else - */ -void initPlugins(); - /** * Load the configuration (from `nix.conf`, `NIX_CONFIG`, etc.) into the * given configuration object. diff --git a/src/libstore/local-fs-store.cc b/src/libstore/local-fs-store.cc index 5449b20eb3b..a8b23996159 100644 --- a/src/libstore/local-fs-store.cc +++ b/src/libstore/local-fs-store.cc @@ -5,25 +5,88 @@ #include "globals.hh" #include "compression.hh" #include "derivations.hh" +#include "config-parse-impl.hh" namespace nix { -LocalFSStoreConfig::LocalFSStoreConfig(PathView rootDir, const Params & params) +LocalFSStore::Config::Descriptions::Descriptions() + : Store::Config::Descriptions{Store::Config::descriptions} + , LocalFSStoreConfigT{ + .rootDir = { + .name = "root", + .description = "Directory prefixed to all other paths.", + }, + .stateDir = { + .name = "state", + .description = "Directory where Nix will store state.", + }, + .logDir = { + .name = "log", + .description = "directory where Nix will store log files.", + }, + .realStoreDir{ + .name = "real", + .description = "Physical path of the Nix store.", + }, + } +{} + +const LocalFSStore::Config::Descriptions LocalFSStore::Config::descriptions{}; + +LocalFSStoreConfigT LocalFSStoreConfig::defaults( + const Store::Config & storeConfig, + const std::optional rootDir) +{ + return { + .rootDir = {.value = std::nullopt }, + .stateDir = {.value = rootDir ? *rootDir + "/nix/var/nix" : settings.nixStateDir }, + .logDir = {.value = rootDir ? *rootDir + "/nix/var/log/nix" : settings.nixLogDir }, + .realStoreDir = {.value = rootDir ? *rootDir + "/nix/store" : storeConfig.storeDir }, + }; +} + +/** + * @param rootDir Fallback if not in `params` + */ +auto localFSStoreConfig( + const Store::Config & storeConfig, + const std::optional _rootDir, + const StoreReference::Params & params) +{ + const auto & descriptions = LocalFSStore::Config::descriptions; + + auto rootDir = descriptions.rootDir.parseConfig(params) + .value_or(config::JustValue{.value = std::move(_rootDir)}); + + auto defaults = LocalFSStore::Config::defaults(storeConfig, rootDir.value); + + return LocalFSStoreConfigT{ + CONFIG_ROW(rootDir), + CONFIG_ROW(stateDir), + CONFIG_ROW(logDir), + CONFIG_ROW(realStoreDir), + }; +} + +LocalFSStore::Config::LocalFSStoreConfig(const StoreReference::Params & params) + : StoreConfig{params} + , LocalFSStoreConfigT{localFSStoreConfig(*this, std::nullopt, params)} +{ +} + +LocalFSStore::Config::LocalFSStoreConfig(PathView rootDir, const StoreReference::Params & params) : StoreConfig(params) - // Default `?root` from `rootDir` if non set - // FIXME don't duplicate description once we don't have root setting - , rootDir{ - this, - !rootDir.empty() && params.count("root") == 0 - ? (std::optional{rootDir}) - : std::nullopt, - "root", - "Directory prefixed to all other paths."} + , LocalFSStoreConfigT{localFSStoreConfig( + *this, + // Default `?root` from `rootDir` if non set + !rootDir.empty() ? (std::optional{rootDir}) : std::nullopt, + params)} { } -LocalFSStore::LocalFSStore(const Params & params) - : Store(params) +LocalFSStore::LocalFSStore(const Config & config) + : LocalFSStore::Config{config} + , Store{static_cast(*this)} { } @@ -97,8 +160,8 @@ std::optional LocalFSStore::getBuildLogExact(const StorePath & path Path logPath = j == 0 - ? fmt("%s/%s/%s/%s", logDir, drvsLogDir, baseName.substr(0, 2), baseName.substr(2)) - : fmt("%s/%s/%s", logDir, drvsLogDir, baseName); + ? fmt("%s/%s/%s/%s", logDir.get(), drvsLogDir, baseName.substr(0, 2), baseName.substr(2)) + : fmt("%s/%s/%s", logDir.get(), drvsLogDir, baseName); Path logBz2Path = logPath + ".bz2"; if (pathExists(logPath)) diff --git a/src/libstore/local-fs-store.hh b/src/libstore/local-fs-store.hh index 9bb569f0b25..654dd210f40 100644 --- a/src/libstore/local-fs-store.hh +++ b/src/libstore/local-fs-store.hh @@ -7,9 +7,39 @@ namespace nix { -struct LocalFSStoreConfig : virtual StoreConfig +template class F> +struct LocalFSStoreConfigT { - using StoreConfig::StoreConfig; + const F> rootDir; + + const F stateDir; + + const F logDir; + + const F realStoreDir; +}; + +struct LocalFSStoreConfig : + virtual Store::Config, + LocalFSStoreConfigT +{ + struct Descriptions : + virtual Store::Config::Descriptions, + LocalFSStoreConfigT + { + Descriptions(); + }; + + static const Descriptions descriptions; + + /** + * The other defaults depend on the choice of `storeDir` and `rootDir` + */ + static LocalFSStoreConfigT defaults( + const Store::Config &, + const std::optional rootDir); + + LocalFSStoreConfig(const StoreReference::Params &); /** * Used to override the `root` settings. Can't be done via modifying @@ -18,38 +48,22 @@ struct LocalFSStoreConfig : virtual StoreConfig * * @todo Make this less error-prone with new store settings system. */ - LocalFSStoreConfig(PathView path, const Params & params); - - const OptionalPathSetting rootDir{this, std::nullopt, - "root", - "Directory prefixed to all other paths."}; - - const PathSetting stateDir{this, - rootDir.get() ? *rootDir.get() + "/nix/var/nix" : settings.nixStateDir, - "state", - "Directory where Nix will store state."}; - - const PathSetting logDir{this, - rootDir.get() ? *rootDir.get() + "/nix/var/log/nix" : settings.nixLogDir, - "log", - "directory where Nix will store log files."}; - - const PathSetting realStoreDir{this, - rootDir.get() ? *rootDir.get() + "/nix/store" : storeDir, "real", - "Physical path of the Nix store."}; + LocalFSStoreConfig(PathView path, const StoreReference::Params & params); }; -class LocalFSStore : public virtual LocalFSStoreConfig, - public virtual Store, - public virtual GcStore, - public virtual LogStore +struct LocalFSStore : + virtual LocalFSStoreConfig, + virtual Store, + virtual GcStore, + virtual LogStore { -public: + using Config = LocalFSStoreConfig; + inline static std::string operationName = "Local Filesystem Store"; const static std::string drvsLogDir; - LocalFSStore(const Params & params); + LocalFSStore(const Config & params); void narFromPath(const StorePath & path, Sink & sink) override; ref getFSAccessor(bool requireValidPath = true) override; diff --git a/src/libstore/local-store.cc b/src/libstore/local-store.cc index 819cee34532..2dfc78ebda3 100644 --- a/src/libstore/local-store.cc +++ b/src/libstore/local-store.cc @@ -17,6 +17,9 @@ #include "posix-source-accessor.hh" #include "keys.hh" #include "users.hh" +#include "config-parse-impl.hh" +#include "store-open.hh" +#include "store-registration.hh" #include #include @@ -56,12 +59,50 @@ namespace nix { -LocalStoreConfig::LocalStoreConfig( +LocalStore::Config::Descriptions::Descriptions() + : Store::Config::Descriptions{Store::Config::descriptions} + , LocalFSStore::Config::Descriptions{LocalFSStore::Config::descriptions} + , LocalStoreConfigT{ + .requireSigs = { + .name = "require-sigs", + .description = "Whether store paths copied into this store should have a trusted signature.", + }, + .readOnly = { + .name = "read-only", + .description = R"( + Allow this store to be opened when its [database](@docroot@/glossary.md#gloss-nix-database) is on a read-only filesystem. + + Normally Nix will attempt to open the store database in read-write mode, even for querying (when write access is not needed), causing it to fail if the database is on a read-only filesystem. + + Enable read-only mode to disable locking and open the SQLite database with the [`immutable` parameter](https://www.sqlite.org/c3ref/open.html) set. + + > **Warning** + > Do not use this unless the filesystem is read-only. + > + > Using it when the filesystem is writable can cause incorrect query results or corruption errors if the database is changed by another process. + > While the filesystem the database resides on might appear to be read-only, consider whether another user or system might have write access to it. + )", + }, + } +{} + +const LocalStore::Config::Descriptions LocalStore::Config::descriptions{}; + +decltype(LocalStore::Config::defaults) LocalStore::Config::defaults = { + .requireSigs = {.value = settings.requireSigs }, + .readOnly = {.value = false }, +}; + +LocalStore::Config::LocalStoreConfig( std::string_view scheme, std::string_view authority, - const Params & params) - : StoreConfig(params) - , LocalFSStoreConfig(authority, params) + const StoreReference::Params & params) + : Store::Config(params) + , LocalFSStore::Config(authority, params) + , LocalStoreConfigT{ + CONFIG_ROW(requireSigs), + CONFIG_ROW(readOnly), + } { } @@ -72,6 +113,11 @@ std::string LocalStoreConfig::doc() ; } +std::shared_ptr LocalStore::Config::openStore() +{ + return std::make_shared(*this); +} + struct LocalStore::State::Stmts { /* Some precompiled SQLite statements. */ SQLiteStmt RegisterValidPath; @@ -192,15 +238,12 @@ void migrateCASchema(SQLite& db, Path schemaPath, AutoCloseFD& lockFd) } } -LocalStore::LocalStore( - std::string_view scheme, - PathView path, - const Params & params) - : StoreConfig(params) - , LocalFSStoreConfig(path, params) - , LocalStoreConfig(scheme, path, params) - , Store(params) - , LocalFSStore(params) +LocalStore::LocalStore(const Config & config) + : Store::Config{config} + , LocalFSStore::Config{config} + , LocalStore::Config{config} + , Store{static_cast(*this)} + , LocalFSStore{static_cast(*this)} , dbDir(stateDir + "/db") , linksDir(realStoreDir + "/.links") , reservedPath(dbDir + "/reserved") @@ -477,12 +520,6 @@ LocalStore::LocalStore( } -LocalStore::LocalStore(const Params & params) - : LocalStore("local", "", params) -{ -} - - AutoCloseFD LocalStore::openGCLock() { Path fnGCLock = stateDir + "/gc.lock"; @@ -1512,7 +1549,7 @@ LocalStore::VerificationResult LocalStore::verifyAllValidPaths(RepairFlag repair database and the filesystem) in the loop below, in order to catch invalid states. */ - for (auto & i : std::filesystem::directory_iterator{realStoreDir.to_string()}) { + for (auto & i : std::filesystem::directory_iterator{realStoreDir.get()}) { checkInterrupt(); try { storePathsInStoreDir.insert({i.path().filename().string()}); @@ -1802,6 +1839,6 @@ std::optional LocalStore::getVersion() return nixVersion; } -static RegisterStoreImplementation regLocalStore; +static RegisterStoreImplementation regLocalStore; } // namespace nix diff --git a/src/libstore/local-store.hh b/src/libstore/local-store.hh index a03cfc03b30..13e26922c0a 100644 --- a/src/libstore/local-store.hh +++ b/src/libstore/local-store.hh @@ -34,36 +34,40 @@ struct OptimiseStats uint64_t bytesFreed = 0; }; -struct LocalStoreConfig : virtual LocalFSStoreConfig +template class F> +struct LocalStoreConfigT { - using LocalFSStoreConfig::LocalFSStoreConfig; + const F requireSigs; - LocalStoreConfig( - std::string_view scheme, - std::string_view authority, - const Params & params); + const F readOnly; +}; - Setting requireSigs{this, - settings.requireSigs, - "require-sigs", - "Whether store paths copied into this store should have a trusted signature."}; +struct LocalStoreConfig : + virtual Store::Config, + virtual LocalFSStore::Config, + LocalStoreConfigT +{ + struct Descriptions : + virtual Store::Config::Descriptions, + virtual LocalFSStore::Config::Descriptions, + LocalStoreConfigT + { + Descriptions(); + }; - Setting readOnly{this, - false, - "read-only", - R"( - Allow this store to be opened when its [database](@docroot@/glossary.md#gloss-nix-database) is on a read-only filesystem. + static const Descriptions descriptions; - Normally Nix will attempt to open the store database in read-write mode, even for querying (when write access is not needed), causing it to fail if the database is on a read-only filesystem. + /** + * The other defaults depend on the choice of `storeDir` and `rootDir` + */ + static LocalStoreConfigT defaults; - Enable read-only mode to disable locking and open the SQLite database with the [`immutable` parameter](https://www.sqlite.org/c3ref/open.html) set. + LocalStoreConfig(const StoreReference::Params &); - > **Warning** - > Do not use this unless the filesystem is read-only. - > - > Using it when the filesystem is writable can cause incorrect query results or corruption errors if the database is changed by another process. - > While the filesystem the database resides on might appear to be read-only, consider whether another user or system might have write access to it. - )"}; + LocalStoreConfig( + std::string_view scheme, + std::string_view authority, + const StoreReference::Params & params); const std::string name() override { return "Local Store"; } @@ -71,12 +75,19 @@ struct LocalStoreConfig : virtual LocalFSStoreConfig { return {"local"}; } std::string doc() override; + + std::shared_ptr openStore() override; }; -class LocalStore : public virtual LocalStoreConfig - , public virtual IndirectRootStore - , public virtual GcStore +class LocalStore : + public virtual LocalStoreConfig, + public virtual IndirectRootStore, + public virtual GcStore { +public: + + using Config = LocalStoreConfig; + private: /** @@ -144,11 +155,7 @@ public: * Initialise the local store, upgrading the schema if * necessary. */ - LocalStore(const Params & params); - LocalStore( - std::string_view scheme, - PathView path, - const Params & params); + LocalStore(const Config & params); ~LocalStore(); diff --git a/src/libstore/machines.hh b/src/libstore/machines.hh index 983652d5f8b..671de208468 100644 --- a/src/libstore/machines.hh +++ b/src/libstore/machines.hh @@ -1,6 +1,8 @@ #pragma once ///@file +#include + #include "ref.hh" #include "store-reference.hh" diff --git a/src/libstore/path.cc b/src/libstore/path.cc index f2fa91cb2e2..3e9d054778c 100644 --- a/src/libstore/path.cc +++ b/src/libstore/path.cc @@ -89,7 +89,7 @@ StorePath StoreDirConfig::parseStorePath(std::string_view path) const canonPath(std::string(path)) #endif ; - if (dirOf(p) != storeDir.get()) + if (dirOf(p) != storeDir) throw BadStorePath("path '%s' is not in the Nix store", p); return StorePath(baseNameOf(p)); } diff --git a/src/libstore/profiles.hh b/src/libstore/profiles.hh index b10a72330b4..63aa9972c9b 100644 --- a/src/libstore/profiles.hh +++ b/src/libstore/profiles.hh @@ -84,7 +84,7 @@ typedef std::list Generations; */ std::pair> findGenerations(Path profile); -class LocalFSStore; +struct LocalFSStore; /** * Create a new generation of the given profile diff --git a/src/libstore/remote-store.cc b/src/libstore/remote-store.cc index d749ccd0a35..6e8931ca2e4 100644 --- a/src/libstore/remote-store.cc +++ b/src/libstore/remote-store.cc @@ -128,7 +128,7 @@ void RemoteStore::setOptions(Connection & conn) overrides.erase(settings.useSubstitutes.name); overrides.erase(loggerSettings.showTrace.name); overrides.erase(experimentalFeatureSettings.experimentalFeatures.name); - overrides.erase(settings.pluginFiles.name); + overrides.erase("plugin-files"); conn.to << overrides.size(); for (auto & i : overrides) conn.to << i.first << i.second.value; diff --git a/src/libstore/remote-store.hh b/src/libstore/remote-store.hh index 4e18962683d..f19aa0fc57b 100644 --- a/src/libstore/remote-store.hh +++ b/src/libstore/remote-store.hh @@ -35,14 +35,15 @@ struct RemoteStoreConfig : virtual StoreConfig * \todo RemoteStore is a misnomer - should be something like * DaemonStore. */ -class RemoteStore : public virtual RemoteStoreConfig, +struct RemoteStore : + public virtual RemoteStoreConfig, public virtual Store, public virtual GcStore, public virtual LogStore { -public: + using Config = RemoteStoreConfig; - RemoteStore(const Params & params); + RemoteStore(const Config & config); /* Implementations of abstract store API methods. */ diff --git a/src/libstore/store-api.cc b/src/libstore/store-api.cc index 2c4dee518b4..c4bb9d594b0 100644 --- a/src/libstore/store-api.cc +++ b/src/libstore/store-api.cc @@ -5,6 +5,7 @@ #include "realisation.hh" #include "derivations.hh" #include "store-api.hh" +#include "store-open.hh" #include "util.hh" #include "nar-info-disk-cache.hh" #include "thread-pool.hh" @@ -18,6 +19,7 @@ #include "worker-protocol.hh" #include "signals.hh" #include "users.hh" +#include "config-parse-impl.hh" #include #include @@ -28,7 +30,6 @@ using json = nlohmann::json; namespace nix { - bool StoreDirConfig::isInStore(PathView path) const { return isInDir(path, storeDir); @@ -188,6 +189,77 @@ std::pair StoreDirConfig::computeStorePath( } +Store::Config::Descriptions::Descriptions() + : StoreDirConfig::Descriptions{StoreDirConfig::descriptions} + , StoreConfigT{ + .pathInfoCacheSize = { + .name = "path-info-cache-size", + .description = "Size of the in-memory store path metadata cache.", + }, + .isTrusted = { + .name = "trusted", + .description = R"( + Whether paths from this store can be used as substitutes + even if they are not signed by a key listed in the + [`trusted-public-keys`](@docroot@/command-ref/conf-file.md#conf-trusted-public-keys) + setting. + )", + }, + .priority = { + .name = "priority", + .description = R"( + Priority of this store when used as a [substituter](@docroot@/command-ref/conf-file.md#conf-substituters). + A lower value means a higher priority. + )", + }, + .wantMassQuery = { + .name = "want-mass-query", + .description = R"( + Whether this store can be queried efficiently for path validity when used as a [substituter](@docroot@/command-ref/conf-file.md#conf-substituters). + )", + }, + + .systemFeatures = { + .name = "system-features", + .description = R"( + Optional [system features](@docroot@/command-ref/conf-file.md#conf-system-features) available on the system this store uses to build derivations. + + Example: `"kvm"` + )", + // The default value is CPU- and OS-specific, and thus + // unsuitable to be rendered in the documentation. + .documentDefault = false, + }, + } +{ +} + + +const Store::Config::Descriptions Store::Config::descriptions{}; + + +decltype(Store::Config::defaults) Store::Config::defaults = { + .pathInfoCacheSize = { .value = 65536 }, + .isTrusted = { .value = false }, + .priority = { .value = 0 }, + .wantMassQuery = { .value = false }, + .systemFeatures = { .value = Store::Config::getDefaultSystemFeatures() }, +}; + + +Store::Config::StoreConfig(const StoreReference::Params & params) + : StoreDirConfig{params} + , StoreConfigT{ + CONFIG_ROW(pathInfoCacheSize), + CONFIG_ROW(isTrusted), + CONFIG_ROW(priority), + CONFIG_ROW(wantMassQuery), + CONFIG_ROW(systemFeatures), + } +{ +} + + StorePath Store::addToStore( std::string_view name, const SourcePath & path, @@ -419,7 +491,7 @@ ValidPathInfo Store::addToStoreSlow( return info; } -StringSet StoreConfig::getDefaultSystemFeatures() +StringSet Store::Config::getDefaultSystemFeatures() { auto res = settings.systemFeatures.get(); @@ -432,8 +504,8 @@ StringSet StoreConfig::getDefaultSystemFeatures() return res; } -Store::Store(const Params & params) - : StoreConfig(params) +Store::Store(const Store::Config & config) + : Store::Config(config) , state({(size_t) pathInfoCacheSize}) { assertLibStoreInitialized(); @@ -1267,102 +1339,3 @@ Derivation Store::readInvalidDerivation(const StorePath & drvPath) { return readDerivationCommon(*this, drvPath, false); } } - - -#include "local-store.hh" -#include "uds-remote-store.hh" - - -namespace nix { - -ref openStore(const std::string & uri, - const Store::Params & extraParams) -{ - return openStore(StoreReference::parse(uri, extraParams)); -} - -ref openStore(StoreReference && storeURI) -{ - auto & params = storeURI.params; - - auto store = std::visit(overloaded { - [&](const StoreReference::Auto &) -> std::shared_ptr { - auto stateDir = getOr(params, "state", settings.nixStateDir); - if (access(stateDir.c_str(), R_OK | W_OK) == 0) - return std::make_shared(params); - else if (pathExists(settings.nixDaemonSocketFile)) - return std::make_shared(params); - #if __linux__ - else if (!pathExists(stateDir) - && params.empty() - && !isRootUser() - && !getEnv("NIX_STORE_DIR").has_value() - && !getEnv("NIX_STATE_DIR").has_value()) - { - /* If /nix doesn't exist, there is no daemon socket, and - we're not root, then automatically set up a chroot - store in ~/.local/share/nix/root. */ - auto chrootStore = getDataDir() + "/nix/root"; - if (!pathExists(chrootStore)) { - try { - createDirs(chrootStore); - } catch (SystemError & e) { - return std::make_shared(params); - } - warn("'%s' does not exist, so Nix will use '%s' as a chroot store", stateDir, chrootStore); - } else - debug("'%s' does not exist, so Nix will use '%s' as a chroot store", stateDir, chrootStore); - return std::make_shared("local", chrootStore, params); - } - #endif - else - return std::make_shared(params); - }, - [&](const StoreReference::Specified & g) { - for (auto implem : *Implementations::registered) - if (implem.uriSchemes.count(g.scheme)) - return implem.create(g.scheme, g.authority, params); - - throw Error("don't know how to open Nix store with scheme '%s'", g.scheme); - }, - }, storeURI.variant); - - experimentalFeatureSettings.require(store->experimentalFeature()); - store->warnUnknownSettings(); - store->init(); - - return ref { store }; -} - -std::list> getDefaultSubstituters() -{ - static auto stores([]() { - std::list> stores; - - StringSet done; - - auto addStore = [&](const std::string & uri) { - if (!done.insert(uri).second) return; - try { - stores.push_back(openStore(uri)); - } catch (Error & e) { - logWarning(e.info()); - } - }; - - for (auto uri : settings.substituters.get()) - addStore(uri); - - stores.sort([](ref & a, ref & b) { - return a->priority < b->priority; - }); - - return stores; - } ()); - - return stores; -} - -std::vector * Implementations::registered = 0; - -} diff --git a/src/libstore/store-api.hh b/src/libstore/store-api.hh index a4d2138f66d..09352e601ba 100644 --- a/src/libstore/store-api.hh +++ b/src/libstore/store-api.hh @@ -9,7 +9,6 @@ #include "lru-cache.hh" #include "sync.hh" #include "globals.hh" -#include "config.hh" #include "path-info.hh" #include "repair-flag.hh" #include "store-dir-config.hh" @@ -41,7 +40,7 @@ namespace nix { * 2. A class `Foo : virtual Store, virtual FooConfig` that contains the * implementation of the store. * - * This class is expected to have a constructor `Foo(const Params & params)` + * This class is expected to have a constructor `Foo(const StoreReference::Params & params)` * that calls `StoreConfig(params)` (otherwise you're gonna encounter an * `assertion failure` when trying to instantiate it). * @@ -97,16 +96,42 @@ struct KeyedBuildResult; typedef std::map> StorePathCAMap; -struct StoreConfig : public StoreDirConfig +template class F> +struct StoreConfigT { - using StoreDirConfig::StoreDirConfig; + const F pathInfoCacheSize; - StoreConfig() = delete; + const F isTrusted; - static StringSet getDefaultSystemFeatures(); + F priority; + + F wantMassQuery; + + F systemFeatures; +}; + +StoreConfigT parseStoreConfig(const StoreReference::Params &); +struct StoreConfig : + StoreDirConfig, + StoreConfigT +{ + struct Descriptions : + StoreDirConfig::Descriptions, + StoreConfigT + { + Descriptions(); + }; + + static const StoreConfigT defaults; + + static const Descriptions descriptions; + + StoreConfig(const StoreReference::Params &); virtual ~StoreConfig() { } + static StringSet getDefaultSystemFeatures(); + /** * The name of this type of store. */ @@ -129,42 +154,19 @@ struct StoreConfig : public StoreDirConfig return std::nullopt; } - const Setting pathInfoCacheSize{this, 65536, "path-info-cache-size", - "Size of the in-memory store path metadata cache."}; - - const Setting isTrusted{this, false, "trusted", - R"( - Whether paths from this store can be used as substitutes - even if they are not signed by a key listed in the - [`trusted-public-keys`](@docroot@/command-ref/conf-file.md#conf-trusted-public-keys) - setting. - )"}; - - Setting priority{this, 0, "priority", - R"( - Priority of this store when used as a [substituter](@docroot@/command-ref/conf-file.md#conf-substituters). - A lower value means a higher priority. - )"}; - - Setting wantMassQuery{this, false, "want-mass-query", - R"( - Whether this store can be queried efficiently for path validity when used as a [substituter](@docroot@/command-ref/conf-file.md#conf-substituters). - )"}; - - Setting systemFeatures{this, getDefaultSystemFeatures(), - "system-features", - R"( - Optional [system features](@docroot@/command-ref/conf-file.md#conf-system-features) available on the system this store uses to build derivations. - - Example: `"kvm"` - )", - {}, - // Don't document the machine-specific default value - false}; + /** + * Open a store of the type corresponding to this configuration + * type. + */ + virtual std::shared_ptr openStore() = 0; }; class Store : public std::enable_shared_from_this, public virtual StoreConfig { +public: + + using Config = StoreConfig; + protected: struct PathInfoCacheValue { @@ -203,14 +205,9 @@ protected: std::shared_ptr diskCache; - Store(const Params & params); + Store(const Store::Config & config); public: - /** - * Perform any necessary effectful operation to make the store up and - * running - */ - virtual void init() {}; virtual ~Store() { } @@ -855,74 +852,6 @@ StorePath resolveDerivedPath(Store &, const SingleDerivedPath &, Store * evalSto OutputPathMap resolveDerivedPath(Store &, const DerivedPath::Built &, Store * evalStore = nullptr); -/** - * @return a Store object to access the Nix store denoted by - * ‘uri’ (slight misnomer...). - */ -ref openStore(StoreReference && storeURI); - - -/** - * Opens the store at `uri`, where `uri` is in the format expected by `StoreReference::parse` - - */ -ref openStore(const std::string & uri = settings.storeUri.get(), - const Store::Params & extraParams = Store::Params()); - - -/** - * @return the default substituter stores, defined by the - * ‘substituters’ option and various legacy options. - */ -std::list> getDefaultSubstituters(); - -struct StoreFactory -{ - std::set uriSchemes; - /** - * The `authorityPath` parameter is `/`, or really - * whatever comes after `://` and before `?`. - */ - std::function ( - std::string_view scheme, - std::string_view authorityPath, - const Store::Params & params)> create; - std::function ()> getConfig; -}; - -struct Implementations -{ - static std::vector * registered; - - template - static void add() - { - if (!registered) registered = new std::vector(); - StoreFactory factory{ - .uriSchemes = TConfig::uriSchemes(), - .create = - ([](auto scheme, auto uri, auto & params) - -> std::shared_ptr - { return std::make_shared(scheme, uri, params); }), - .getConfig = - ([]() - -> std::shared_ptr - { return std::make_shared(StringMap({})); }) - }; - registered->push_back(factory); - } -}; - -template -struct RegisterStoreImplementation -{ - RegisterStoreImplementation() - { - Implementations::add(); - } -}; - - /** * Display a set of paths in human-readable form (i.e., between quotes * and separated by commas). diff --git a/src/libstore/store-dir-config.cc b/src/libstore/store-dir-config.cc index 775f678be4a..a51962d7fa9 100644 --- a/src/libstore/store-dir-config.cc +++ b/src/libstore/store-dir-config.cc @@ -1,18 +1,11 @@ #include "store-dir-config.hh" -#include "config-parse.hh" +#include "config-parse-impl.hh" #include "util.hh" namespace nix { -const StoreDirConfigT storeDirConfigDefaults = { - .storeDir = - { - .value = settings.nixStore, - }, -}; - -const StoreDirConfigT storeDirConfigDescriptions = { - .storeDir = +const StoreDirConfigT StoreDirConfig::descriptions = { + ._storeDir = { .name = "store", .description = R"( @@ -23,18 +16,14 @@ const StoreDirConfigT storeDirConfigDescriptions = { }, }; -StoreDirConfigT parseStoreDirConfig(const StoreReference::Params & params) -{ - constexpr auto & defaults = storeDirConfigDefaults; - constexpr auto & descriptions = storeDirConfigDescriptions; - - return { - CONFIG_ROW(storeDir), - }; -} +const StoreDirConfigT StoreDirConfig::defaults = { + ._storeDir = {.value = settings.nixStore}, +}; StoreDirConfig::StoreDirConfig(const StoreReference::Params & params) - : StoreDirConfigT{parseStoreDirConfig(params)} + : StoreDirConfigT{ + CONFIG_ROW(_storeDir), + } { } diff --git a/src/libstore/store-dir-config.hh b/src/libstore/store-dir-config.hh index e2f61a72995..19391322907 100644 --- a/src/libstore/store-dir-config.hh +++ b/src/libstore/store-dir-config.hh @@ -4,7 +4,7 @@ #include "hash.hh" #include "content-address.hh" #include "store-reference.hh" -#include "config-abstract.hh" +#include "config-parse.hh" #include "globals.hh" #include @@ -22,21 +22,23 @@ MakeError(BadStorePathName, BadStorePath); template class F> struct StoreDirConfigT { - F storeDir; + const F _storeDir; }; -extern const StoreDirConfigT storeDirConfigDefaults; +struct StoreDirConfig : StoreDirConfigT +{ + static const StoreDirConfigT defaults; -extern const StoreDirConfigT storeDirConfigDescriptions; + using Descriptions = StoreDirConfigT; -StoreDirConfigT parseStoreDirConfig(const StoreReference::Params &); + static const Descriptions descriptions; -struct StoreDirConfig : StoreDirConfigT -{ StoreDirConfig(const StoreReference::Params & params); virtual ~StoreDirConfig() = default; + const Path & storeDir = _storeDir.value; + // pure methods StorePath parseStorePath(std::string_view path) const; diff --git a/src/libstore/store-open.hh b/src/libstore/store-open.hh new file mode 100644 index 00000000000..ab6f3cb7739 --- /dev/null +++ b/src/libstore/store-open.hh @@ -0,0 +1,38 @@ +#pragma once +/** + * @file + * + * For opening a store described by an `StoreReference`, which is an "untyped" + * notion which needs to be decoded against a collection of specific + * implementations. + * + * For consumers of the store registration machinery defined in + * `store-registration.hh`. Not needed by store implementation definitions, or + * usages of a given `Store` which will be passed in. + */ + +#include "store-api.hh" + +namespace nix { + +/** + * @return a Store object to access the Nix store denoted by + * ‘uri’ (slight misnomer...). + */ +ref openStore(StoreReference && storeURI); + +/** + * Opens the store at `uri`, where `uri` is in the format expected by `StoreReference::parse` + + */ +ref openStore( + const std::string & uri = settings.storeUri.get(), + const StoreReference::Params & extraParams = StoreReference::Params()); + +/** + * @return the default substituter stores, defined by the + * ‘substituters’ option and various legacy options. + */ +std::list> getDefaultSubstituters(); + +} diff --git a/src/libstore/store-reference.cc b/src/libstore/store-reference.cc index b4968dfadbd..5e5d30fc117 100644 --- a/src/libstore/store-reference.cc +++ b/src/libstore/store-reference.cc @@ -1,5 +1,7 @@ #include +#include + #include "error.hh" #include "url.hh" #include "store-reference.hh" diff --git a/src/libstore/store-reference.hh b/src/libstore/store-reference.hh index 459cea9c20e..c1066296afe 100644 --- a/src/libstore/store-reference.hh +++ b/src/libstore/store-reference.hh @@ -2,6 +2,7 @@ ///@file #include +#include #include "types.hh" @@ -41,7 +42,7 @@ namespace nix { */ struct StoreReference { - using Params = std::map; + using Params = nlohmann::json::object_t; /** * Special store reference `""` or `"auto"` diff --git a/src/libstore/store-registration.cc b/src/libstore/store-registration.cc new file mode 100644 index 00000000000..85ece9623cf --- /dev/null +++ b/src/libstore/store-registration.cc @@ -0,0 +1,95 @@ +#include "store-registration.hh" +#include "store-open.hh" +#include "local-store.hh" +#include "uds-remote-store.hh" + +namespace nix { + +ref openStore(const std::string & uri, const Store::Params & extraParams) +{ + return openStore(StoreReference::parse(uri, extraParams)); +} + +ref openStore(StoreReference && storeURI) +{ + auto & params = storeURI.params; + + auto store = std::visit( + overloaded{ + [&](const StoreReference::Auto &) -> std::shared_ptr { + auto stateDir = getOr(params, "state", settings.nixStateDir); + if (access(stateDir.c_str(), R_OK | W_OK) == 0) + return std::make_shared(params); + else if (pathExists(settings.nixDaemonSocketFile)) + return std::make_shared(params); +#if __linux__ + else if ( + !pathExists(stateDir) && params.empty() && !isRootUser() && !getEnv("NIX_STORE_DIR").has_value() + && !getEnv("NIX_STATE_DIR").has_value()) { + /* If /nix doesn't exist, there is no daemon socket, and + we're not root, then automatically set up a chroot + store in ~/.local/share/nix/root. */ + auto chrootStore = getDataDir() + "/nix/root"; + if (!pathExists(chrootStore)) { + try { + createDirs(chrootStore); + } catch (SystemError & e) { + return std::make_shared(params); + } + warn("'%s' does not exist, so Nix will use '%s' as a chroot store", stateDir, chrootStore); + } else + debug("'%s' does not exist, so Nix will use '%s' as a chroot store", stateDir, chrootStore); + return std::make_shared("local", chrootStore, params); + } +#endif + else + return std::make_shared(params); + }, + [&](const StoreReference::Specified & g) { + for (auto implem : *Implementations::registered) + if (implem.uriSchemes.count(g.scheme)) + return implem.create(g.scheme, g.authority, params); + + throw Error("don't know how to open Nix store with scheme '%s'", g.scheme); + }, + }, + storeURI.variant); + + experimentalFeatureSettings.require(store->experimentalFeature()); + store->warnUnknownSettings(); + store->init(); + + return ref{store}; +} + +std::vector * Implementations::registered = 0; + +std::list> getDefaultSubstituters() +{ + static auto stores([]() { + std::list> stores; + + StringSet done; + + auto addStore = [&](const std::string & uri) { + if (!done.insert(uri).second) + return; + try { + stores.push_back(openStore(uri)); + } catch (Error & e) { + logWarning(e.info()); + } + }; + + for (auto uri : settings.substituters.get()) + addStore(uri); + + stores.sort([](ref & a, ref & b) { return a->priority < b->priority; }); + + return stores; + }()); + + return stores; +} + +} diff --git a/src/libstore/store-registration.hh b/src/libstore/store-registration.hh new file mode 100644 index 00000000000..8d57f4a587a --- /dev/null +++ b/src/libstore/store-registration.hh @@ -0,0 +1,56 @@ +#pragma once +/** + * @file + * + * Infrastructure for "registering" store implementations. Used by the + * store implementation definitions themselves but not by consumers of + * those implementations. + */ + +#include "store-api.hh" + +namespace nix { + +struct StoreFactory +{ + std::set uriSchemes; + /** + * The `authorityPath` parameter is `/`, or really + * whatever comes after `://` and before `?`. + */ + std::function( + std::string_view scheme, std::string_view authorityPath, const StoreReference::Params & params)> + parseConfig; + const Store::Config::Descriptions & configDescriptions; +}; + +struct Implementations +{ + static std::vector * registered; + + template + static void add() + { + if (!registered) + registered = new std::vector(); + StoreFactory factory{ + .uriSchemes = T::Config::uriSchemes(), + .parseConfig = ([](auto scheme, auto uri, auto & params) -> std::shared_ptr { + return std::make_shared(scheme, uri, params); + }), + .configDescriptions = T::Config::descriptions, + }; + registered->push_back(factory); + } +}; + +template +struct RegisterStoreImplementation +{ + RegisterStoreImplementation() + { + Implementations::add(); + } +}; + +} diff --git a/src/libstore/worker-protocol-connection.cc b/src/libstore/worker-protocol-connection.cc index 072bae8da82..3a640051ebc 100644 --- a/src/libstore/worker-protocol-connection.cc +++ b/src/libstore/worker-protocol-connection.cc @@ -152,7 +152,6 @@ WorkerProto::BasicClientConnection::handshake(BufferedSink & to, Source & from, throw Error("Nix daemon protocol version not supported"); if (GET_PROTOCOL_MINOR(daemonVersion) < 10) throw Error("the Nix daemon version is too old"); - to << localVersion; return std::min(daemonVersion, localVersion); } diff --git a/src/libutil/config-abstract.hh b/src/libutil/config-abstract.hh index 081478dc195..10c98f0a73f 100644 --- a/src/libutil/config-abstract.hh +++ b/src/libutil/config-abstract.hh @@ -1,7 +1,7 @@ #pragma once ///@type -namespace nix { +namespace nix::config { template struct JustValue @@ -23,10 +23,9 @@ struct JustValue }; template -struct SettingInfo +auto && operator<<(auto && str, const JustValue & opt) { - std::string name; - std::string description; -}; + return str << opt.get(); +} } diff --git a/src/libutil/meson.build b/src/libutil/meson.build index 1337b700141..76ca662cc6e 100644 --- a/src/libutil/meson.build +++ b/src/libutil/meson.build @@ -215,6 +215,7 @@ headers = [config_h] + files( 'source-accessor.hh', 'source-path.hh', 'split.hh', + 'std-hash.hh', 'strings.hh', 'strings-inline.hh', 'suggestions.hh', diff --git a/src/libutil/source-path.hh b/src/libutil/source-path.hh index 1e96b72e52d..fc2288f747a 100644 --- a/src/libutil/source-path.hh +++ b/src/libutil/source-path.hh @@ -8,8 +8,7 @@ #include "ref.hh" #include "canon-path.hh" #include "source-accessor.hh" - -#include // for boost::hash_combine +#include "std-hash.hh" namespace nix { diff --git a/src/libutil/std-hash.hh b/src/libutil/std-hash.hh new file mode 100644 index 00000000000..c359d11ca63 --- /dev/null +++ b/src/libutil/std-hash.hh @@ -0,0 +1,24 @@ +#pragma once + +//!@file Hashing utilities for use with unordered_map, etc. (ie low level implementation logic, not domain logic like +//! Nix hashing) + +#include + +namespace nix { + +/** + * hash_combine() from Boost. Hash several hashable values together + * into a single hash. + */ +inline void hash_combine(std::size_t & seed) {} + +template +inline void hash_combine(std::size_t & seed, const T & v, Rest... rest) +{ + std::hash hasher; + seed ^= hasher(v) + 0x9e3779b9 + (seed << 6) + (seed >> 2); + hash_combine(seed, rest...); +} + +} // namespace nix diff --git a/src/libutil/util.hh b/src/libutil/util.hh index 83b42a5283e..877d1527945 100644 --- a/src/libutil/util.hh +++ b/src/libutil/util.hh @@ -375,18 +375,4 @@ inline std::string operator + (std::string_view s1, const char * s2) return s; } -/** - * hash_combine() from Boost. Hash several hashable values together - * into a single hash. - */ -inline void hash_combine(std::size_t & seed) { } - -template -inline void hash_combine(std::size_t & seed, const T & v, Rest... rest) -{ - std::hash hasher; - seed ^= hasher(v) + 0x9e3779b9 + (seed<<6) + (seed>>2); - hash_combine(seed, rest...); -} - } diff --git a/src/nix-build/nix-build.cc b/src/nix-build/nix-build.cc index 4d373ef8d4e..9e5dfd1a04c 100644 --- a/src/nix-build/nix-build.cc +++ b/src/nix-build/nix-build.cc @@ -11,7 +11,7 @@ #include "current-process.hh" #include "parsed-derivations.hh" -#include "store-api.hh" +#include "store-open.hh" #include "local-fs-store.hh" #include "globals.hh" #include "realisation.hh" @@ -183,6 +183,9 @@ static void main_nix_build(int argc, char * * argv) struct MyArgs : LegacyArgs, MixEvalArgs { using LegacyArgs::LegacyArgs; + void setBaseDir(Path baseDir) { + commandBaseDir = baseDir; + } }; MyArgs myArgs(myName, [&](Strings::iterator & arg, const Strings::iterator & end) { @@ -290,6 +293,9 @@ static void main_nix_build(int argc, char * * argv) state->repair = myArgs.repair; if (myArgs.repair) buildMode = bmRepair; + if (inShebang && compatibilitySettings.nixShellShebangArgumentsRelativeToScript) { + myArgs.setBaseDir(absPath(dirOf(script))); + } auto autoArgs = myArgs.getAutoArgs(*state); auto autoArgsWithInNixShell = autoArgs; @@ -334,8 +340,13 @@ static void main_nix_build(int argc, char * * argv) exprs = {state->parseStdin()}; else for (auto i : remainingArgs) { + auto baseDir = inShebang && !packages ? absPath(dirOf(script)) : i; + if (fromArgs) - exprs.push_back(state->parseExprFromString(std::move(i), state->rootPath("."))); + exprs.push_back(state->parseExprFromString( + std::move(i), + (inShebang && compatibilitySettings.nixShellShebangArgumentsRelativeToScript) ? lookupFileArg(*state, baseDir) : state->rootPath(".") + )); else { auto absolute = i; try { diff --git a/src/nix-channel/nix-channel.cc b/src/nix-channel/nix-channel.cc index 9f7f557b59d..028a7f36f24 100644 --- a/src/nix-channel/nix-channel.cc +++ b/src/nix-channel/nix-channel.cc @@ -2,7 +2,7 @@ #include "shared.hh" #include "globals.hh" #include "filetransfer.hh" -#include "store-api.hh" +#include "store-open.hh" #include "legacy.hh" #include "eval-settings.hh" // for defexpr #include "users.hh" diff --git a/src/nix-collect-garbage/nix-collect-garbage.cc b/src/nix-collect-garbage/nix-collect-garbage.cc index 91209c97898..9830b1058a3 100644 --- a/src/nix-collect-garbage/nix-collect-garbage.cc +++ b/src/nix-collect-garbage/nix-collect-garbage.cc @@ -1,6 +1,6 @@ #include "file-system.hh" #include "signals.hh" -#include "store-api.hh" +#include "store-open.hh" #include "store-cast.hh" #include "gc-store.hh" #include "profiles.hh" diff --git a/src/nix-copy-closure/nix-copy-closure.cc b/src/nix-copy-closure/nix-copy-closure.cc index b64af758fcb..d910adcf876 100644 --- a/src/nix-copy-closure/nix-copy-closure.cc +++ b/src/nix-copy-closure/nix-copy-closure.cc @@ -1,6 +1,6 @@ #include "shared.hh" #include "realisation.hh" -#include "store-api.hh" +#include "store-open.hh" #include "legacy.hh" using namespace nix; diff --git a/src/nix-env/nix-env.cc b/src/nix-env/nix-env.cc index 5e170c99d37..056e9025800 100644 --- a/src/nix-env/nix-env.cc +++ b/src/nix-env/nix-env.cc @@ -9,7 +9,7 @@ #include "profiles.hh" #include "path-with-outputs.hh" #include "shared.hh" -#include "store-api.hh" +#include "store-open.hh" #include "local-fs-store.hh" #include "user-env.hh" #include "value-to-json.hh" diff --git a/src/nix-instantiate/nix-instantiate.cc b/src/nix-instantiate/nix-instantiate.cc index c4854951124..5474ed0b7cb 100644 --- a/src/nix-instantiate/nix-instantiate.cc +++ b/src/nix-instantiate/nix-instantiate.cc @@ -8,7 +8,7 @@ #include "signals.hh" #include "value-to-xml.hh" #include "value-to-json.hh" -#include "store-api.hh" +#include "store-open.hh" #include "local-fs-store.hh" #include "common-eval-args.hh" #include "legacy.hh" diff --git a/src/nix-store/nix-store.cc b/src/nix-store/nix-store.cc index f073074e851..40d20b37690 100644 --- a/src/nix-store/nix-store.cc +++ b/src/nix-store/nix-store.cc @@ -2,6 +2,7 @@ #include "derivations.hh" #include "dotgraph.hh" #include "globals.hh" +#include "store-open.hh" #include "store-cast.hh" #include "local-fs-store.hh" #include "log-store.hh" diff --git a/src/nix/flake.cc b/src/nix/flake.cc index 3f9f8f99b06..26e553bde07 100644 --- a/src/nix/flake.cc +++ b/src/nix/flake.cc @@ -8,7 +8,7 @@ #include "flake/flake.hh" #include "get-drvs.hh" #include "signals.hh" -#include "store-api.hh" +#include "store-open.hh" #include "derivations.hh" #include "outputs-spec.hh" #include "attr-path.hh" diff --git a/src/nix/flake.md b/src/nix/flake.md index 2f43d02640d..46d5a3867bf 100644 --- a/src/nix/flake.md +++ b/src/nix/flake.md @@ -259,6 +259,8 @@ Currently the `type` attribute can be one of the following: `.tgz`, `.tar.gz`, `.tar.xz`, `.tar.bz2` or `.tar.zst`), then the `tarball+` can be dropped. + This can also be used to set the location of gitea/forgejo branches. [See here](@docroot@/protocols/tarball-fetcher.md#gitea-and-forgejo-support) + * `file`: Plain files or directory tarballs, either over http(s) or from the local disk. diff --git a/src/nix/log.cc b/src/nix/log.cc index 7f590c708f6..efa4a584ccc 100644 --- a/src/nix/log.cc +++ b/src/nix/log.cc @@ -1,7 +1,7 @@ #include "command.hh" #include "common-args.hh" #include "shared.hh" -#include "store-api.hh" +#include "store-open.hh" #include "log-store.hh" #include "progress-bar.hh" diff --git a/src/nix/main.cc b/src/nix/main.cc index 00ad6fe2c97..0d839425afb 100644 --- a/src/nix/main.cc +++ b/src/nix/main.cc @@ -8,7 +8,8 @@ #include "globals.hh" #include "legacy.hh" #include "shared.hh" -#include "store-api.hh" +#include "store-open.hh" +#include "store-registration.hh" #include "filetransfer.hh" #include "finally.hh" #include "loggers.hh" diff --git a/src/nix/make-content-addressed.cc b/src/nix/make-content-addressed.cc index d9c988a9f5d..0c3efd7f499 100644 --- a/src/nix/make-content-addressed.cc +++ b/src/nix/make-content-addressed.cc @@ -1,5 +1,5 @@ #include "command.hh" -#include "store-api.hh" +#include "store-open.hh" #include "make-content-addressed.hh" #include "common-args.hh" diff --git a/src/nix/prefetch.cc b/src/nix/prefetch.cc index db7d9e4efe6..fa087430f86 100644 --- a/src/nix/prefetch.cc +++ b/src/nix/prefetch.cc @@ -1,7 +1,7 @@ #include "command.hh" #include "common-args.hh" #include "shared.hh" -#include "store-api.hh" +#include "store-open.hh" #include "filetransfer.hh" #include "finally.hh" #include "progress-bar.hh" diff --git a/src/nix/repl.cc b/src/nix/repl.cc index a2f3e033e72..8f74149042d 100644 --- a/src/nix/repl.cc +++ b/src/nix/repl.cc @@ -1,6 +1,7 @@ #include "eval.hh" #include "eval-settings.hh" #include "globals.hh" +#include "store-open.hh" #include "command.hh" #include "installable-value.hh" #include "repl.hh" diff --git a/src/nix/sigs.cc b/src/nix/sigs.cc index 1e277cbbee7..5a8693ab6c4 100644 --- a/src/nix/sigs.cc +++ b/src/nix/sigs.cc @@ -1,7 +1,7 @@ #include "signals.hh" #include "command.hh" #include "shared.hh" -#include "store-api.hh" +#include "store-open.hh" #include "thread-pool.hh" #include "progress-bar.hh" diff --git a/src/nix/unix/daemon.cc b/src/nix/unix/daemon.cc index 4a7997b1fb0..8b618c19e2b 100644 --- a/src/nix/unix/daemon.cc +++ b/src/nix/unix/daemon.cc @@ -7,6 +7,7 @@ #include "local-store.hh" #include "remote-store.hh" #include "remote-store-connection.hh" +#include "store-open.hh" #include "serialise.hh" #include "archive.hh" #include "globals.hh" @@ -239,7 +240,7 @@ static PeerInfo getPeerInfo(int remote) */ static ref openUncachedStore() { - Store::Params params; // FIXME: get params from somewhere + StoreReference::Params params; // FIXME: get params from somewhere // Disable caching since the client already does that. params["path-info-cache-size"] = "0"; return openStore(settings.storeUri, params); diff --git a/src/nix/verify.cc b/src/nix/verify.cc index 124a05bed2c..3e2253fdf04 100644 --- a/src/nix/verify.cc +++ b/src/nix/verify.cc @@ -1,6 +1,6 @@ #include "command.hh" #include "shared.hh" -#include "store-api.hh" +#include "store-open.hh" #include "thread-pool.hh" #include "signals.hh" #include "keys.hh" diff --git a/tests/functional/flakes/run.sh b/tests/functional/flakes/run.sh index c2c345ed2cb..61af6049a00 100755 --- a/tests/functional/flakes/run.sh +++ b/tests/functional/flakes/run.sh @@ -29,5 +29,26 @@ nix run --no-write-lock-file .#pkgAsPkg ! nix run --no-write-lock-file .#pkgAsApp || fail "'nix run' shouldn’t accept an 'app' defined under 'packages'" ! nix run --no-write-lock-file .#appAsPkg || fail "elements of 'apps' should be of type 'app'" +# Test that we're not setting any more environment variables than necessary. +# For instance, we might set an environment variable temporarily to affect some +# initialization or whatnot, but this must not leak into the environment of the +# command being run. +env > $TEST_ROOT/expected-env +nix run -f shell-hello.nix env > $TEST_ROOT/actual-env +# Remove/reset variables we expect to be different. +# - PATH is modified by nix shell +# - _ is set by bash and is expected to differ because it contains the original command +# - __CF_USER_TEXT_ENCODING is set by macOS and is beyond our control +sed -i \ + -e 's/PATH=.*/PATH=.../' \ + -e 's/_=.*/_=.../' \ + -e '/^__CF_USER_TEXT_ENCODING=.*$/d' \ + $TEST_ROOT/expected-env $TEST_ROOT/actual-env +sort $TEST_ROOT/expected-env | uniq > $TEST_ROOT/expected-env.sorted +# nix run appears to clear _. I don't understand why. Is this ok? +echo "_=..." >> $TEST_ROOT/actual-env +sort $TEST_ROOT/actual-env | uniq > $TEST_ROOT/actual-env.sorted +diff $TEST_ROOT/expected-env.sorted $TEST_ROOT/actual-env.sorted + clearStore diff --git a/tests/functional/nix-shell.sh b/tests/functional/nix-shell.sh index 65ff279f868..b9625eb666f 100755 --- a/tests/functional/nix-shell.sh +++ b/tests/functional/nix-shell.sh @@ -65,6 +65,25 @@ chmod a+rx $TEST_ROOT/shell.shebang.sh output=$($TEST_ROOT/shell.shebang.sh abc def) [ "$output" = "foo bar abc def" ] +# Test nix-shell shebang mode with an alternate working directory +sed -e "s|@ENV_PROG@|$(type -P env)|" shell.shebang.expr > $TEST_ROOT/shell.shebang.expr +chmod a+rx $TEST_ROOT/shell.shebang.expr +# Should fail due to expressions using relative path +! $TEST_ROOT/shell.shebang.expr bar +cp shell.nix config.nix $TEST_ROOT +# Should succeed +echo "cwd: $PWD" +output=$($TEST_ROOT/shell.shebang.expr bar) +[ "$output" = foo ] + +# Test nix-shell shebang mode with an alternate working directory +sed -e "s|@ENV_PROG@|$(type -P env)|" shell.shebang.legacy.expr > $TEST_ROOT/shell.shebang.legacy.expr +chmod a+rx $TEST_ROOT/shell.shebang.legacy.expr +# Should fail due to expressions using relative path +mkdir -p "$TEST_ROOT/somewhere-unrelated" +output="$(cd "$TEST_ROOT/somewhere-unrelated"; $TEST_ROOT/shell.shebang.legacy.expr bar;)" +[[ $(realpath "$output") = $(realpath "$TEST_ROOT/somewhere-unrelated") ]] + # Test nix-shell shebang mode again with metacharacters in the filename. # First word of filename is chosen to not match any file in the test root. sed -e "s|@ENV_PROG@|$(type -P env)|" shell.shebang.sh > $TEST_ROOT/spaced\ \\\'\"shell.shebang.sh diff --git a/tests/functional/repl/pretty-print-idempotent.expected b/tests/functional/repl/pretty-print-idempotent.expected new file mode 100644 index 00000000000..f38b9b56969 --- /dev/null +++ b/tests/functional/repl/pretty-print-idempotent.expected @@ -0,0 +1,29 @@ +Nix +Type :? for help. +Added variables. + +{ homepage = "https://example.com"; } + +{ homepage = "https://example.com"; } + +{ + layerOne = { ... }; +} + +{ + layerOne = { ... }; +} + +[ "https://example.com" ] + +[ "https://example.com" ] + +[ + [ ... ] +] + +[ + [ ... ] +] + + diff --git a/tests/functional/repl/pretty-print-idempotent.in b/tests/functional/repl/pretty-print-idempotent.in new file mode 100644 index 00000000000..5f865316fdf --- /dev/null +++ b/tests/functional/repl/pretty-print-idempotent.in @@ -0,0 +1,9 @@ +:l pretty-print-idempotent.nix +oneDeep +oneDeep +twoDeep +twoDeep +oneDeepList +oneDeepList +twoDeepList +twoDeepList diff --git a/tests/functional/repl/pretty-print-idempotent.nix b/tests/functional/repl/pretty-print-idempotent.nix new file mode 100644 index 00000000000..68929f387e4 --- /dev/null +++ b/tests/functional/repl/pretty-print-idempotent.nix @@ -0,0 +1,19 @@ +{ + oneDeep = { + homepage = "https://" + "example.com"; + }; + twoDeep = { + layerOne = { + homepage = "https://" + "example.com"; + }; + }; + + oneDeepList = [ + ("https://" + "example.com") + ]; + twoDeepList = [ + [ + ("https://" + "example.com") + ] + ]; +} diff --git a/tests/functional/shell-hello.nix b/tests/functional/shell-hello.nix index c46fdec8a8c..c920d7cb459 100644 --- a/tests/functional/shell-hello.nix +++ b/tests/functional/shell-hello.nix @@ -55,4 +55,26 @@ rec { chmod +x $out/bin/hello ''; }; + + # execs env from PATH, so that we can probe the environment + # does not allow arguments, because we don't need them + env = mkDerivation { + name = "env"; + outputs = [ "out" ]; + buildCommand = + '' + mkdir -p $out/bin + + cat > $out/bin/env <&2 + exit 1 + fi + exec env + EOF + chmod +x $out/bin/env + ''; + }; + } diff --git a/tests/functional/shell.nix b/tests/functional/shell.nix index 750cdf0bcaa..9cae14b780f 100644 --- a/tests/functional/shell.nix +++ b/tests/functional/shell.nix @@ -46,6 +46,7 @@ let pkgs = rec { ASCII_PERCENT = "%"; ASCII_AT = "@"; TEST_inNixShell = if inNixShell then "true" else "false"; + FOO = fooContents; inherit stdenv; outputs = ["dev" "out"]; } // { diff --git a/tests/functional/shell.sh b/tests/functional/shell.sh index b4c03f5477c..c2ac3b24d4a 100755 --- a/tests/functional/shell.sh +++ b/tests/functional/shell.sh @@ -23,6 +23,25 @@ nix shell -f shell-hello.nix hello-symlink -c hello | grep 'Hello World' # Test that symlinks outside of the store don't work. expect 1 nix shell -f shell-hello.nix forbidden-symlink -c hello 2>&1 | grepQuiet "is not in the Nix store" +# Test that we're not setting any more environment variables than necessary. +# For instance, we might set an environment variable temporarily to affect some +# initialization or whatnot, but this must not leak into the environment of the +# command being run. +env > $TEST_ROOT/expected-env +nix shell -f shell-hello.nix hello -c env > $TEST_ROOT/actual-env +# Remove/reset variables we expect to be different. +# - PATH is modified by nix shell +# - _ is set by bash and is expectedf to differ because it contains the original command +# - __CF_USER_TEXT_ENCODING is set by macOS and is beyond our control +sed -i \ + -e 's/PATH=.*/PATH=.../' \ + -e 's/_=.*/_=.../' \ + -e '/^__CF_USER_TEXT_ENCODING=.*$/d' \ + $TEST_ROOT/expected-env $TEST_ROOT/actual-env +sort $TEST_ROOT/expected-env > $TEST_ROOT/expected-env.sorted +sort $TEST_ROOT/actual-env > $TEST_ROOT/actual-env.sorted +diff $TEST_ROOT/expected-env.sorted $TEST_ROOT/actual-env.sorted + if isDaemonNewer "2.20.0pre20231220"; then # Test that command line attribute ordering is reflected in the PATH # https://github.com/NixOS/nix/issues/7905 diff --git a/tests/functional/shell.shebang.expr b/tests/functional/shell.shebang.expr new file mode 100755 index 00000000000..c602dedbf03 --- /dev/null +++ b/tests/functional/shell.shebang.expr @@ -0,0 +1,9 @@ +#! @ENV_PROG@ nix-shell +#! nix-shell "{ script, path, ... }: assert path == ./shell.nix; script { }" +#! nix-shell --no-substitute +#! nix-shell --expr +#! nix-shell --arg script "import ./shell.nix" +#! nix-shell --arg path "./shell.nix" +#! nix-shell -A shellDrv +#! nix-shell -i bash +echo "$FOO" diff --git a/tests/functional/shell.shebang.legacy.expr b/tests/functional/shell.shebang.legacy.expr new file mode 100755 index 00000000000..490542f430b --- /dev/null +++ b/tests/functional/shell.shebang.legacy.expr @@ -0,0 +1,10 @@ +#! @ENV_PROG@ nix-shell +#! nix-shell "{ script, path, ... }: assert path == ./shell.nix; script { fooContents = toString ./.; }" +#! nix-shell --no-substitute +#! nix-shell --expr +#! nix-shell --arg script "import ((builtins.getEnv ''TEST_ROOT'')+''/shell.nix'')" +#! nix-shell --arg path "./shell.nix" +#! nix-shell -A shellDrv +#! nix-shell -i bash +#! nix-shell --option nix-shell-shebang-arguments-relative-to-script false +echo "$FOO" diff --git a/tests/unit/libstore-support/tests/libstore.hh b/tests/unit/libstore-support/tests/libstore.hh index 84be52c230b..5710a12d9b6 100644 --- a/tests/unit/libstore-support/tests/libstore.hh +++ b/tests/unit/libstore-support/tests/libstore.hh @@ -5,6 +5,7 @@ #include #include "store-api.hh" +#include "store-open.hh" namespace nix { diff --git a/tests/unit/libstore/local-store.cc b/tests/unit/libstore/local-store.cc index e85cf0b83f0..e22f54bd45d 100644 --- a/tests/unit/libstore/local-store.cc +++ b/tests/unit/libstore/local-store.cc @@ -2,12 +2,6 @@ #include "local-store.hh" -// Needed for template specialisations. This is not good! When we -// overhaul how store configs work, this should be fixed. -#include "args.hh" -#include "config-impl.hh" -#include "abstract-setting-to-json.hh" - namespace nix { TEST(LocalStore, constructConfig_rootQueryParam) diff --git a/tests/unit/libutil-support/tests/string_callback.cc b/tests/unit/libutil-support/tests/string_callback.cc index 2d0e0dad08f..7a13bd4ff9c 100644 --- a/tests/unit/libutil-support/tests/string_callback.cc +++ b/tests/unit/libutil-support/tests/string_callback.cc @@ -2,9 +2,10 @@ namespace nix::testing { -void observe_string_cb(const char * start, unsigned int n, std::string * user_data) +void observe_string_cb(const char * start, unsigned int n, void * user_data) { - *user_data = std::string(start); + auto user_data_casted = reinterpret_cast(user_data); + *user_data_casted = std::string(start); } } diff --git a/tests/unit/libutil-support/tests/string_callback.hh b/tests/unit/libutil-support/tests/string_callback.hh index a02ea3a1b69..9a7e8d85dab 100644 --- a/tests/unit/libutil-support/tests/string_callback.hh +++ b/tests/unit/libutil-support/tests/string_callback.hh @@ -3,14 +3,13 @@ namespace nix::testing { -void observe_string_cb(const char * start, unsigned int n, std::string * user_data); +void observe_string_cb(const char * start, unsigned int n, void * user_data); inline void * observe_string_cb_data(std::string & out) { return (void *) &out; }; -#define OBSERVE_STRING(str) \ - (nix_get_string_callback) nix::testing::observe_string_cb, nix::testing::observe_string_cb_data(str) +#define OBSERVE_STRING(str) nix::testing::observe_string_cb, nix::testing::observe_string_cb_data(str) }