(ErlangError) Erlang error: :emfile during mix.bulk_analyze

[kitplummer]

Takes a while, but eventually something runs out. Too many simultaneous open files perhaps. Happening in HPC - so its likely not possible to extend limits, not that that would be the right answer anywho.

Function: &AnalyzerModule.analyze/3
Args: ["git://github.com/creationix/stack.git", "mix task", %{counter: #PID<0.406.0>, types: false}]
** (EXIT from #PID<0.92.0>) an exception was raised:
** (ErlangError) Erlang error: :emfile
:erlang.open_port({:spawn_executable, '/usr/bin/du'}, [:use_stdio, :exit_status, :binary, :hide, {:args, ["-sh", "/gv1/users/cplummer8/tmp/lei-1621522952-7624-vj3fxo/stack.git"]}])
(elixir 1.10.2) lib/system.ex:802: System.cmd/3
lib/git_module.ex:306: GitModule.get_repo_size/1
lib/analyzer_module.ex:120: AnalyzerModule.analyze/3
(elixir 1.10.2) lib/task/supervised.ex:90: Task.Supervised.invoke_mfa/2
(elixir 1.10.2) lib/task/supervised.ex:35: Task.Supervised.reply/5
(stdlib 3.11) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

[kitplummer]

Need to replace du with git count-objects which returns:

❯ git count-objects
898 objects, 3768 kilobytes

So, will simply need to parse out the objects and size. Should be pretty simple.

[kitplummer]

Confirmed the Erlang error does indicate the file limit problem - too many concurrent processes running causes the issue. We want the concurrency so this has to get replaced.