lib.types: Add deferredModule #163617

roberth · 2022-03-10T21:49:19Z

Description of changes

Great for submodule defaults. Example: network.defaults in NixOps.

Accepts modules, returns a list of modules, with a good default module location.

I'll quote the docs below. Also look at the example.

types.deferredModule

Whereas submodule represents an option tree, deferredModule represents a module value, such as a module file or a configuration.

It can be set multiple times.

Module authors can use its value, which is always a list of module values, in imports or in submoduleWith's modules parameter.
Note that imports must be evaluated before the module fixpoint. Because of this, deferred modules can only be imported into "other" fixpoints, such as submodules.

One use case for this type is the type of a "default" module that allow the user to affect all submodules in an attrsOf submodule at once. This is more convenient and discoverable than expecting the module user to type-merge with the attrsOf submodule option.

Refs NixOS/nixops#1508

Things done

roberth · 2022-04-11T15:26:30Z

cc @infinisil

roberth · 2022-05-18T16:50:23Z

ping @infinisil

infinisil · 2022-05-19T13:35:40Z

lib/types.nix

+    deferredModule = mkOptionType {
+      name = "deferredModule";
+      description = "module";
+      check = t: isAttrs t || isFunction t;


Shouldn't this also allow paths?

And how about also allowing lists of modules, instead of just a single one

paths

done

lists

This should be covered by lib.mkMerge. I don't expect lists to be common enough to warrant the duplication.

Hmm, it bothers me a bit that the result of this type is a list, but the input is a single element. This is in contrast to essentially all other types whose input and output types are generally the same.

Also, if the input type is also a list (which I think also makes more sense, since it's how imports, the modules argument for evalModules/submoduleWith works), then it might make sense to use the listOf type to implement it.

Which then brings me to my general dislike of lists, the main problem being that it's not possible to remove entries once added. I guess in this case it's somewhat fine though, because I believe disabledModules can be used to essentially remove modules from the list, though finding out how to actually refer to modules in the list is pretty hard.

This reminds me of Html in Elm, which is also an inherently monoidal type that they're constantly putting in lists (the free monoid) for no good reason. Similar to their [Html] -> Html operation (<div> or <span>), our concat operation (x: { imports = x; }) is also leaky. It leaks through in error messages with anon--something-something strings.

I've embraced the leakiness and I've work with the rules:

keep the conversions to a minimum (they're leaky)

keep the interface simple

we don't have a use case for defining lists of modules, and it's still covered by mkMerge

submodule doesn't support lists, so why should deferredModule?

It'd be a different store if lists of modules were always considered to be valid modules. This would include the submodule argument, submodule definitions, imports items, evalModules { modules } items, etc.

my general dislike of lists

I share the dislike, as they imply ordering, which is often not necessary or desirable in a declarative system. In this case though, they are only as unnecessary and undesirable as in the rest of the modules system - not something we can expect to solve here.

disabledModules

The best solution to removals is not to include the things in the first place.
disabledModules should operate just fine on the parent module and on path-imported ones. Anonymous ones will be equally problematic as anonymous submodules.

though finding out how to actually refer to modules in the list is pretty hard.

Do you mean list indexing? That's not something I'd think of, but I guess someone could try - and eventually fail horribly.

I guess this might be reason to make it slightly harder? Although x.imports !! y isn't much of an obstacle over x !! y., especially compared to writing the function that only removes one element. I don't think we have that in lib?

I suppose I'm not entirely opposed to returning a single module. Given my argumentation, do you still think I should do it? (or something else)

I suppose I'm not entirely opposed to returning a single module. Given my argumentation, do you still think I should do it? (or something else)

I haven't considered that, but I really do like that idea! By returning a single module we avoid lists in the type sense while keeping the input and output types the same, that's a 👍 👍 from me! It also aligns with the type name of types.deferredModule instead of types.deferredModules

infinisil · 2022-05-19T13:41:40Z

Nice change :D

roberth · 2022-05-25T16:00:38Z

I've found an additional use case (flake.parts) where I need "static" modules with option docs support, similar to submoduleWith.

I've noticed that we can't preserve the location info of such modules yet, just like the modules passed to submoduleWith don't support this. We can solve this issue separately, perhaps even by using unsafeGetAttrPos, which would be more valuable than _file anyway.

infinisil

Other than two nits this looks good. I tested this with https://github.com/infinisil/dream2nix-modules, a demo for using the module system for dream2nix created with feedback from @DavHau at Zurihac. Turns out deferredModule is just what was needed there

infinisil · 2022-06-14T17:30:16Z

lib/modules.nix

@@ -464,7 +464,9 @@ rec {
        disabledModules = m.disabledModules or [];
        imports = m.require or [] ++ m.imports or [];
        options = {};
-        config = addFreeformType (addMeta (removeAttrs m ["_file" "key" "disabledModules" "require" "imports" "freeformType"]));
+        config =
+          lib.throwIfNot (isAttrs m) "module ${file} (${key}) does not look like a module."


I think this check should be done earlier, ideally at the start of this function, since m is always required to be an attribute set, not just for this specific case here.

nixos/doc/manual/development/option-types.section.md

roberth · 2022-06-14T21:22:14Z

Turns out deferredModule is just what was needed there

Nice! I've already used it in flake.parts perSystem, NixOps 2, and I will use it again in #176557

`m` must always be an attrset at this point. It is basically always evaluated. This will make it throw when any of the attrs is accessed, rather than just `config`. We assume that this will improve the error message in more scenarios.

infinisil

Looking good!

roberth · 2022-06-15T11:31:27Z

Thanks @infinisil!

aakropotkin · 2022-12-30T16:03:38Z

@roberth do you have any usage examples? This looks useful but the only examples I could find were two mentions in a test case that didn't reveal much.

infinisil · 2023-01-18T20:42:36Z

@aakropotkin You can look at the test case in this PR

roberth · 2023-01-20T00:11:51Z

flake-parts perSystem is one. It's for modeling anything that's a function of system, as used by the new Nix Flakes CLI.

roberth requested review from edolstra, nbp and infinisil as code owners March 10, 2022 21:49

github-actions bot added 6.topic: nixos 8.has: documentation labels Mar 10, 2022

roberth mentioned this pull request Mar 10, 2022

eval-machine-info.nix: ported to modules NixOS/nixops#1508

Open

9 tasks

roberth added the 6.topic: module system About "NixOS" module system internals label Mar 10, 2022

roberth force-pushed the lib-types-deferredModule branch from e31e2b8 to c5a2153 Compare March 10, 2022 21:53

ofborg bot added 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 labels Mar 10, 2022

roberth force-pushed the lib-types-deferredModule branch from c5a2153 to fcb8eef Compare April 11, 2022 15:18

infinisil requested changes May 19, 2022

View reviewed changes

roberth requested a review from infinisil May 19, 2022 15:00

roberth mentioned this pull request May 20, 2022

merge perSystem system argument with the rest hercules-ci/flake-parts#18

Closed

roberth force-pushed the lib-types-deferredModule branch from 2d59a20 to 23df9a1 Compare May 25, 2022 16:12

infinisil reviewed Jun 14, 2022

View reviewed changes

roberth added 6 commits June 14, 2022 23:01

lib.types: Add deferredModule

4746f6d

lib/test/modules.sh: Test deferredModule error location file

38b7709

lib.types.deferredModule: Improve reported location

a2c2956

lib.types.deferredModule: Allow path-typed module references

781c2e0

lib.types: Add deferredModuleWith

19a069a

lib.deferredModule: Make it properly singular

dfd98a5

roberth force-pushed the lib-types-deferredModule branch from 4d94100 to 49f374c Compare June 14, 2022 21:17

roberth added 2 commits June 14, 2022 23:23

nixos/doc: Hold off on NixOps 2 info until released

d9dccae

roberth force-pushed the lib-types-deferredModule branch from 49f374c to d9dccae Compare June 14, 2022 21:23

roberth requested a review from infinisil June 14, 2022 21:50

roberth mentioned this pull request Jun 14, 2022

nixosTest matrix (prev also modularity, option docs) #176557

Draft

33 tasks

infinisil approved these changes Jun 15, 2022

View reviewed changes

infinisil merged commit 8f8db59 into NixOS:master Jun 15, 2022

Atry mentioned this pull request May 1, 2024

Implement a interface-like system nix-community/NixNG#3

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

lib.types: Add deferredModule #163617

lib.types: Add deferredModule #163617

roberth commented Mar 10, 2022 •

edited

Loading

roberth commented Apr 11, 2022

roberth commented May 18, 2022

infinisil May 19, 2022

infinisil May 19, 2022

roberth May 19, 2022

infinisil May 25, 2022

roberth May 25, 2022

infinisil Jun 13, 2022

infinisil commented May 19, 2022

roberth commented May 25, 2022

infinisil left a comment

infinisil Jun 14, 2022

roberth commented Jun 14, 2022

infinisil left a comment

roberth commented Jun 15, 2022

aakropotkin commented Dec 30, 2022

infinisil commented Jan 18, 2023

roberth commented Jan 20, 2023

lib.types: Add deferredModule #163617

lib.types: Add deferredModule #163617

Conversation

roberth commented Mar 10, 2022 • edited Loading

Description of changes

Things done

roberth commented Apr 11, 2022

roberth commented May 18, 2022

infinisil May 19, 2022

Choose a reason for hiding this comment

infinisil May 19, 2022

Choose a reason for hiding this comment

roberth May 19, 2022

Choose a reason for hiding this comment

infinisil May 25, 2022

Choose a reason for hiding this comment

roberth May 25, 2022

Choose a reason for hiding this comment

infinisil Jun 13, 2022

Choose a reason for hiding this comment

infinisil commented May 19, 2022

roberth commented May 25, 2022

infinisil left a comment

Choose a reason for hiding this comment

infinisil Jun 14, 2022

Choose a reason for hiding this comment

roberth commented Jun 14, 2022

infinisil left a comment

Choose a reason for hiding this comment

roberth commented Jun 15, 2022

aakropotkin commented Dec 30, 2022

infinisil commented Jan 18, 2023

roberth commented Jan 20, 2023

roberth commented Mar 10, 2022 •

edited

Loading