Inconsistent fs.rm behavior with "force" option on Windows vs mac/linux

[VincentBailly]

Version

16.17.0

Platform

Linux my-VM 5.15.0-1022-azure #27~20.04.1-Ubuntu SMP Mon Oct 17 02:03:50 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Subsystem

node:fs

What steps will reproduce the bug?

touch foo
node -e 'require("fs").promises.rm("foo/bar", { force: true })'

How often does it reproduce? Is there a required condition?

No condition, it reproduces on mac and on Linux.

What is the expected behavior?

Same behavior as on Windows, the function call succeeds and no exception is thrown.

What do you see instead?

Unhandled exception

node:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[Error: ENOTDIR: not a directory, stat 'foo/bar'] {
  errno: -20,
  code: 'ENOTDIR',
  syscall: 'stat',
  path: 'foo/bar'
}

Additional information

No response

[dnalborczyk]

only tested this on a mac, but the behavior described above should be correct, as the path foo/bar does not exist. if this works on windows, I would think that it would need to be fixed for that platform, not the other way around.

@VincentBailly are you sure you used the same paths on windows and linux?

[VincentBailly]

@dnalborczyk thank you for the response. The issue was not explicit enough, sorry. Let me clarify.

The issue has to do with the option "force", the documentation says that the "force" option makes the function ignore exceptions when the path does not exist.

https://nodejs.org/docs/latest-v16.x/api/fs.html#fspromisesrmpath-options

[aduh95]

On macOS:

$ touch foo
$ rm -f foo/bar
rm: foo/bar: Not a directory
$ node -e 'fs.promises.rm("foo/bar", {force:true})'
node:internal/process/promises:289
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[Error: ENOTDIR: not a directory, stat 'foo/bar'] {
  errno: -20,
  code: 'ENOTDIR',
  syscall: 'stat',
  path: 'foo/bar'
}

Node.js v20.0.0-pre

It looks like the built-in method also throw in this case. I suggest we update the documentation, I think we should try keeping the behavior consistent with the built-in tool. We should probably fix the Windows behavior then, I don't have a Windows machine to work on that, so if there are any volunteers that's be awesome.

/cc @nodejs/fs

[LiviaMedeiros]

the documentation says that the "force" option makes the function ignore exceptions when the path does not exist.

When true, exceptions will be ignored if path does not exist.

Maybe this part requires rewording to avoid any ambiguity, but it sounds correct to me. force flag tells to ignore ENOENT on the path (i.e. target file or any directory in it), while the error we in this example is ENOTDIR.

We should probably fix the Windows behavior then, I don't have a Windows machine to work on that, so if there are any volunteers that's be awesome.

I'm not sure about direct native equivalent of rm -f in Windows, but it works like this in cygwin:

$ touch foo
$ rm foo/bar
rm: cannot remove 'foo/bar': Not a directory
$ node -e 'require("fs").promises.rm("foo/bar")'
node:internal/process/promises:288
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[Error: ENOENT: no such file or directory, stat 'C:\foo\bar'] {
  errno: -4058,
  code: 'ENOENT',
  syscall: 'stat',
  path: 'C:\\foo\\bar'
}

$ rm -f foo/bar
[no error]
$ node -e 'require("fs").promises.rm("foo/bar", { force: true })'
[no error]

So, behaviour with force: true is consistent. With force: false it throws ENOENT instead.

[VincentBailly]

If this can help: on Linux (Ubuntu 22.04 & bash 5.0.17 & coreutils 8.30) the behavior is inconsistent between rm and nodejs

$ rm -rf foo # cleanup to be sure
$ touch foo
$ rm foo/bar
rm: cannot remove 'foo/bar': Not a directory
$ rm -f foo/bar # no error
$ node -e 'require("fs").promises.rm("foo/bar")' # error expected, same as 'rm'
node:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[Error: ENOTDIR: not a directory, stat 'foo/bar'] {
  errno: -20,
  code: 'ENOTDIR',
  syscall: 'stat',
  path: 'foo/bar'
}
$ node -e 'require("fs").promises.rm("foo/bar", { force: true })' # error non expected, 'rm' command did not error out in same situation.
node:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[Error: ENOTDIR: not a directory, stat 'foo/bar'] {
  errno: -20,
  code: 'ENOTDIR',
  syscall: 'stat',
  path: 'foo/bar'
}

[VincentBailly]

Here is a summary of what different tools do in this scenario:

Tool	behavior	documentation
core-utils (linux)	no error	ignore nonexistent files and arguments
nodejs on linux	error	When true, exceptions will be ignored if path does not exist
-	-	-
Mac os X utilities	error	If the file does not exist, do not display a diagnostic message or modify the exit status to reflect an error.
nodejs on mac	error	When true, exceptions will be ignored if path does not exist
-	-	-
cygwin	no error	ignore nonexistent files,
git bash for windows	no error	ignore nonexistent files and arguments
nodejs on windows	no error	When true, exceptions will be ignored if path does not exist

The NodeJS documentation says that rm is modeled on the standard POSIX rm utility, which says "if file does not exist and -f is specified, skip this file"

[VincentBailly]

Since this is tightly related to the behavior of stat, let's also consider this:

stat tool	behavior
core-utils (linux)	Not a directory
nodejs on linux	Not a directory
-	-
macos utilities	???
nodejs on macos	???
-	-
cygwin	???
git bash for Windows	Not a directory
nodejs on Windows	no such file or directory

[VincentBailly]

It seems that fixing the stat command on Windows will be enough to make rm consistent.

[aduh95]

On macOS:

$ rm -rf foo
$ touch foo
$ stat foo/bar
stat: foo/bar: stat: Not a directory
$ node -e 'fs.statSync("foo/bar")'
node:internal/fs/utils:348
    throw err;
    ^

Error: ENOTDIR: not a directory, stat 'foo/bar'
    at Object.statSync (node:fs:1596:3)
    at [eval]:1:4
    at Script.runInThisContext (node:vm:129:12)
    at Object.runInThisContext (node:vm:313:38)
    at node:internal/process/execution:79:19
    at [eval]-wrapper:6:22
    at evalScript (node:internal/process/execution:78:60)
    at node:internal/main/eval_string:28:3 {
  errno: -20,
  syscall: 'stat',
  code: 'ENOTDIR',
  path: 'foo/bar'
}

Node.js v18.11.0