Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is this repository deprecated/abandoned? #1514

Open
1 task
rennerg opened this issue Mar 6, 2023 · 3 comments
Open
1 task

Is this repository deprecated/abandoned? #1514

rennerg opened this issue Mar 6, 2023 · 3 comments

Comments

@rennerg
Copy link

rennerg commented Mar 6, 2023

Link to bug demonstration repository

Expected Behavior

Observed Behavior

Troubleshooting steps

  • still occurring when I put cache: false in my nyc config

Environment Information

# paste the output here

@Swivelgames
Copy link

Swivelgames commented Mar 13, 2023

I think most people have moved over to using bcoe/c8

Unfortunately, c8 is broken if you're developing a native ESM module. Otherwise, it works wonderfully and is actively developed.

@jcready
Copy link

jcready commented Jul 13, 2023

Otherwise, it works wonderfully

Hard disagree. Regardless of ESM modules c8 (or rather V8) coverage is FAR less accurate with coverage results as it doesn't perform any instrumentation or operate on the AST at all. See jestjs/jest#11188 and vitest-dev/vitest#1252 for more detail, but to summarize:

  • V8 coverage (used by c8) tracks "blocks", not individual statements. These blocks just track character ranges which (with the help of sourcemaps) get converted to ranges in the original source code. Because of this you end up seeing coverage (or lack there of) for things like: empty lines, comments, import statements, typescript type declarations, etc. c8 treats every line as a statement even though that's absolutely not the case.
  • Instanbul/nyc instruments the code by parsing the code into an AST and then inserts its own code to track how many times a block of code was executed. Importantly it tracks implicit else branches by inserting its own else branch with a counter into the AST if one didn't exist. Same thing for binary operand (x && y), ternary (x ? y : z), and assignment logic (x &= y).

If you're depending on coverage thresholds in CI for a public repo you should be extremely cautious of c8 because contributors can easily game the statement/line coverage results by adding whitespace or multiline comments. They can also game the branch coverage by only exercising the truthy side of a branch.

While c8 is faster than nyc, it's only faster because it's sacrifices a LOT of accuracy.

justinmk3 added a commit to aws/aws-toolkit-vscode that referenced this issue Jan 4, 2024
Problem:
`nyc` is no longer maintained: istanbuljs/nyc#1514
and it depends on various outdated packages, .e.g `uuid@3.4.0` (check with `npm ls uuid`).

Solution:
- Remove `nyc`.
- Migrate to `c8`.
justinmk3 added a commit to aws/aws-toolkit-vscode that referenced this issue Jan 5, 2024
Problem:
`nyc` is no longer maintained: istanbuljs/nyc#1514
and it depends on various outdated packages, .e.g `uuid@3.4.0` (check with `npm ls uuid`).

Solution:
- Remove `nyc`.
- Migrate to `c8`.
justinmk3 added a commit to aws/aws-toolkit-vscode that referenced this issue Jan 5, 2024
Problem:
`nyc` is no longer maintained: istanbuljs/nyc#1514
and it depends on various outdated packages, .e.g `uuid@3.4.0` (check with `npm ls uuid`).

Solution:
- Remove `nyc`.
- Migrate to `c8`.
@Swivelgames
Copy link

Swivelgames commented Feb 14, 2024

@jcready Unfortunately, I'm having the same issues. The lack of accuracy makes it really hard to justify continuing to use c8, especially given the lack of traction on c8/34.

It's disappointing that code coverage projects have lost so much traction and attention.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants