What about mutant tests?

[jhomarolo]

From @italojs

What do you think about mutant test?
Some times ago we I had problems with herbs bugs that the unit test don't filtered it, so I was thinking how to mitigate it to happens again, so I would like to propose to use mutant test in herbs libs

The ideia of mutant tests is to test your test, confirm how much your test coverage all situations

example:

// ./src/legalAge.js
module.exports = isLegalAge = age => {
  return age >= 18
}

// ./__tests__/legalAge.spec.js
test('FAIL: must to check if have legal age', () => {
  expect(isLegalAge(17)).toBe(false)
})

the code above have 100% of test coverage, but what happens if some developer change the function code to

module.exports = isLegalAge = age => {
  return age > 18
}

//or 

module.exports = isLegalAge = age => {
  return false
}

my test will pass and I dont will detec the new bug generated by this change because all my testes(100% coverage) is passing.

To solve it, the mutant test modify your code and try to broken your test, if the modified(bugged) code dont broken your tests, you must to improve your scenarios coverage

To fix the code above, the correct tests must to be:

test('FAIL: must to check if 17 is legal age', () => {
  expect(isLegalAge(17)).toBe(false)
})

test('SUCCESS: must to check if 18 is legal age', () => {
  expect(isLegalAge(18)).toBe(true)
})

test('SUCCESS: must to check if gratter then 18 is legal age', () => {
  expect(isLegalAge(19)).toBe(true)
})

some refs:
[pt]

https://github.com/PauloGoncalvesBH/teste-de-mutacao/tree/at-talks
https://www.youtube.com/watch?v=k8Dq8YZgci8&ab_channel=Locaweb
[en]
https://pedrorijo.com/blog/intro-mutation/
http://atodorov.org/blog/2016/12/27/mutation-testing-vs-coverage/
https://medium.com/appsflyer/tests-coverage-is-dead-long-live-mutation-testing-7fd61020330e