[Bug]: treat failure to run the linter differently from linter reporting violations

[alexeagle]

What happened?

We put the exit code of the linter into a file, and treat that as success.

However https://eslint.org/docs/latest/use/command-line-interface#exit-codes for example shows that a linter can distinguish between an error-level violation of a lint rule, and an error that prevents the linter from running. For example, a missing dep on an js_library(srcs = eslint.rc.cjs) target will make a successful lint action, with exit code 2 and no report. The tool that interprets the data doesn't have much to go on, as the error was printed to stderr the first time the tool ran, and isn't repeated in a cache hit nor captured in the report file.

Version

HEAD

How to reproduce

No response

Any other information?

No response

[sallustfire]

Hey @alexeagle, I'm potentially interested in submitting an FR for this issue. I lost a few minutes the other day because of dependency problems in an eslint.config.mjs.

The relevant code seems to be:

if not exit_code:
    ctx.actions.run_shell(
        inputs = _gather_inputs(ctx, srcs),
        outputs = [report],
        tools = [executable._eslint],
        arguments = [args],
        command = executable._eslint.path + " $@ && touch " + report.path,
        env = env,
        mnemonic = _MNEMONIC,
        progress_message = "Linting %{label} with ESLint",
    )
else:
    # Workaround: create an empty report file in case eslint doesn't write one
    # Use `../../..` to return to the execroot?
    args.add_joined(["--node_options", "--require", "../../../" + ctx.file._workaround_17660.path], join_with = "=")

    args.add_all(["--output-file", report.short_path])
    env["JS_BINARY__EXIT_CODE_OUTPUT_FILE"] = exit_code.path

    ctx.actions.run(
        inputs = _gather_inputs(ctx, srcs),
        outputs = [report, exit_code],
        executable = executable._eslint,
        arguments = [args],
        env = env,
        mnemonic = _MNEMONIC,
        progress_message = "Linting %{label} with ESLint",
    )

and then something similar for eslint_fix.

I would think that for exit codes 0 and 1 the behavior would be unchanged, but anything else is a problem with running eslint and the action shouldn't write to the exit code output file nor set the exit code to 0.

The implementation here ties into rules_js internals. Maybe instead of depending on js_binary exit code mechanics for eslint the relevant exit code handling can be done with a wrapper script for rules_lint? Any linter could declare what exit codes occur during normal operation and everything else would be returned as is?