builtins.deepSeq Leads to Segfault

[jeff-hykin]

Describe the bug

Per the docs example I was using deepSeq along with tryEval. However, when given a package as an input (maybe all packages) it crashs the repl with a segfault.

Steps To Reproduce

# NOTE: same behavior on the latest nixpkgs commit (example commit is much older)
# NOTE: also same behavior swapping cowsay with python3Packages.numpy

➜ nix repl -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/aa0e8072a57e879073cee969a780e586dbe57997.tar.gz
Welcome to Nix 2.11.1. Type :? for help.

nix-repl> n = import <nixpkgs> {}
nix-repl> builtins.deepSeq n.cowsay n.cowsay
segmentation fault nix repl -I

➜

Note: on the latest nixpkgs there is a warning (printed many times), however it still ends by crashing the repl

trace: warning: Use `stdenv.tests` instead. `passthru` is a `mkDerivation` detail.

Expected behavior

(I do expect the evaluation to take a very very long time, thats not an issue)

If the process is running out of memory, which is understandable, I would expect a stack overflow error.
Otherwise I would expect some more informative error message explaining why/where it failed.

nix-env --version output

nix-env (Nix) 2.11.1 (Darwin aarch64)

[roberth]

Crashes are not ok, but for completeness:
deepSeq is usually not what you want, because it is incompatible with things like aliases that stick around for deprecations, etc. It also does not support infinite lazy structures, such as pkgs.pkgsStatic.pkgsStatic.pkgsStatic... and what have you.

We do have a known crash on darwin, caused by an incomplete GC/coroutine integration. Could you try with garbage collection disabled?

GC_DONT_GC=1 nix repl

[jeff-hykin]

It also does not support infinite lazy structures

Oh that's really unfortunate. Coincidentally though, I wrote a nix script yesterday that handles infinite lazy structures for recursive BFS exploration (it was only getting hung up by exploring errors, hence the tryEval).
Is there a possibility for me to make a PR that makes deepSeq handle the infinite case? I guess if its a C implementation the nix script won't really matter.

We do have a known crash on darwin

Sadly, still hits a segfault. However, I watched the memory usage and it gets to 6Gb before being killed so it does appear to be in the relm of memory leaks/stack-overflow problems. I imagine it gets to the perl buildInput for cowsay, and then I'm pretty sure perl, or maybe perl packages, has some recursive references.

I also ran it on a linux machine and do get a stack overflow error

...
trace: warning: Use `stdenv.tests` instead. `passthru` is a `mkDerivation` detail.
trace: warning: Use `stdenv.tests` instead. `passthru` is a `mkDerivation` detail.
trace: warning: Use `stdenv.tests` instead. `passthru` is a `mkDerivation` detail.
trace: warning: Use `stdenv.tests` instead. `passthru` is a `mkDerivation` detail.
trace: warning: Use `stdenv.tests` instead. `passthru` is a `mkDerivation` detail.
trace: warning: Use `stdenv.tests` instead. `passthru` is a `mkDerivation` detail.
trace: warning: Use `stdenv.tests` instead. `passthru` is a `mkDerivation` detail.
trace: warning: Use `stdenv.tests` instead. `passthru` is a `mkDerivation` detail.
trace: warning: Use `stdenv.tests` instead. `passthru` is a `mkDerivation` detail.
trace: warning: Use `stdenv.tests` instead. `passthru` is a `mkDerivation` detail.
trace: warning: Use `stdenv.tests` instead. `passthru` is a `mkDerivation` detail.
error: stack overflow (possible infinite recursion)

[roberth]

Is there a possibility for me to make a PR that makes deepSeq handle the infinite case?

My statement was a bit too general. In general, it does not support infinite structures, but it does support looping structures that don't diverge. To illustrate

nix-repl> let loop = { a = loop; }; in builtins.deepSeq loop "ok"                      
"ok"

nix-repl> let diverge = n: { a = diverge (n + 1); }; start = diverge 0; in builtins.deepSeq start "ok"
error: stack overflow (possible infinite recursion)

In the loop case, deepSeq can detect the loop thanks to an internal pointer equality. In diverge it cannot, as every a is new.
deepSeq would need to reason about the expressions that make up every step of the structure, and then decide the undecidable to return before achieving what it set out to achieve.

I think what you're really after is the recurseForDerivations logic that nix-build uses to guide its traversal into nested attribute sets of derivations.

nix/src/libexpr/get-drvs.cc

Line 350 in 90e630a

static void getDerivations(EvalState & state, Value & vIn,

It's not the most elegant formulation, but the gist is to always read the "root" attribute values, and then only recurse into attributes that have recurseForDerivations = true inside of them.

[jeff-hykin]

recurseForDerivations will work for my original task, so thank you for that!

I do want to understand the more general process. It seems really strange to me that cowsay would have a diverging infinite structure inside of it. Are there any good debugging approaches I could use for finding out where it might be looping? (e.g. some sort of trace of the attribute names, so I can see which ones are repeating)

[roberth]

Some attributes behave more like what would be zero-argument methods in a strict language if that makes sense.
(Nix doesn't have methods though; they're fields with a per-instance cost)

Are there any good debugging approaches I could use for finding out where it might be looping?

You could step with the nix debugger (announcement link). This is most useful when it's the Nix code driving the stack use, and not nix-build or perhaps even some primop (builtin) that has the "initiative" to fill the stack.

Nix could perhaps be improved to enforce its own stack limit, which might help, incase that's what's crashing here. #6361 (comment)

Another angle is to use gdb or lldb to load a core dump. My attempts to debug on darwin have been clunky at best. On NixOS it's nix shell nixpkgs#gdb -c coredumpctl debug, although getting debug symbols is still clunky regardless of platform.

[jeff-hykin]

Thanks for the additional detail, looks like thats going to help since I'm exploring older commits before recurseForDerivations was added.

However, for the newer commits I am running into an issue. Parents of nested packages likepkgs.python3Packages have recurseForDerivations = false. Some are true, like perlPackages.recurseForDerivations , but I don't fully understand the pattern. I would've assumed it would be true for python packages could be identified. So I'm not exactly sure how to use this to recursively explore.

This is off the latest commit:
nix repl -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/84554ceb0017f254899c83d2f74c3bc18a4964d9.tar.gz

[Et7f3]

nix eval .#stdenv.all

Seems related (same stack trace). One loop I saw is: tcl depend on stdenv and stdenv depend on tcl. First bootstrapping stage stdenv have this recursion.

[SuperSandro2000]

I also ran it on a linux machine and do get a stack overflow error

I encountered this same pattern when using nix 2.18 for my nix-daemon and 2.17 for my cli which is rather odd to me.

Edit:

I tried running it with --debugger and regardless if I pass --impure or not I get the following which corresponds to this code https://github.com/NixOS/nixpkgs/blob/ccfc0b4b4f6d6051e88229d3189107ad7ac9b3e6/pkgs/top-level/impure.nix#L41C22-L41C26

So the recursion could be in https://github.com/NixOS/nixpkgs/blob/ccfc0b4b4f6d6051e88229d3189107ad7ac9b3e6/pkgs/top-level/impure.nix#L10 ? I am not sure

In my NIX_PATH I do not have nixpkgs-overlays set.

 ▶ /nix/store/11glc2yk4jmycvk42q6kikpdcpbnbf6w-nix-2.17.1/bin/nix eval .#nixosConfigurations.hydrogen.config.system.build.toplevel --show-trace --debugger --impure
error: file 'nixpkgs-overlays' was not found in the Nix search path (add it using $NIX_PATH or -I)

       at «none»:0: (source not available)


This exception occurred in a 'tryEval' call. Use --ignore-try to skip these.

Starting REPL to allow you to inspect the current state of the evaluator.

Welcome to Nix 2.17.1. Type :? for help.

nix-repl>
error: file 'nixpkgs-overlays' was not found in the Nix search path (add it using $NIX_PATH or -I)

       at «none»:0: (source not available)


This exception occurred in a 'tryEval' call. Use --ignore-try to skip these.

Starting REPL to allow you to inspect the current state of the evaluator.

Welcome to Nix 2.17.1. Type :? for help.

nix-repl>

Edit2: that was just a bug while trying to enter the debugger. When temporarily removing the offending code, the original error still appears.