pytest gives "Coverage disabled via --no-cov switch!" warning even --no-cov is not given!!

[iamabigstone]

• a detailed description of the bug or suggestion
  I have not set --no-cov in any cli or configs, but it is given me the warning "Coverage disabled via --no-cov switch!". Even worse, I have enabled coverage in pytest.ini:

[pytest]
addopts = --cov=mypackage -vvv -s

But coverage is not running!

Here is output:

(ENV) $ python -m pytest
=================================================== test session starts ====================================================
platform darwin -- Python 2.7.15, pytest-4.4.1, py-1.8.0, pluggy-0.9.0 -- /Users/local/myrepo_new/ENV/bin/python
cachedir: .pytest_cache
rootdir: /Users/local/myrepo_new, inifile: pytest.ini
plugins: cov-2.6.1
collected 1 item                                                                                                           

test/test_myrepo.py::test_x PASSED
WARNING: Coverage disabled via --no-cov switch!

===================================================== warnings summary =====================================================
ENV/lib/python2.7/site-packages/pytest_cov/plugin.py:253
  /Users/local/myrepo_new/ENV/lib/python2.7/site-packages/pytest_cov/plugin.py:253: PytestWarning: Coverage disabled via --no-cov switch!
    warnings.warn(pytest.PytestWarning(message))

-- Docs: https://docs.pytest.org/en/latest/warnings.html
=========================================== 1 passed, 1 warnings in 0.03 seconds ===========================================

• output of pip list from the virtual environment you are using

(ENV)$ pip list
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Package        Version
-------------- -------
atomicwrites   1.3.0  
attrs          19.1.0 
coverage       4.5.3  
funcsigs       1.0.2  
more-itertools 5.0.0  
pathlib2       2.3.3  
pip            19.1   
pluggy         0.9.0  
py             1.8.0  
pytest         4.4.1  
pytest-cov     2.6.1  
scandir        1.10.0 
setuptools     41.0.1 
six            1.12.0 
wheel          0.33.1 

• pytest and operating system versions

(ENV) $ pytest --version
This is pytest version 4.4.1, imported from /Users/local/myrepo_new/ENV/lib/python2.7/site-packages/pytest.pyc
setuptools registered plugins:
  pytest-cov-2.6.1 at /Users/local/myrepo_new/ENV/lib/python2.7/site-packages/pytest_cov/plugin.py
(ENV)$ plutil -p /System/Library/CoreServices/SystemVersion.plist
{
  "iOSSupportVersion" => "12.0"
  "ProductBuildVersion" => "18D25"
  "ProductCopyright" => "1983-2018 Apple Inc."
  "ProductName" => "Mac OS X"
  "ProductUserVisibleVersion" => "10.14.3"
  "ProductVersion" => "10.14.3"
}

• minimal example if possible
  myrepo_new.zip

[blueyed]

It is more an issue with pytest-cov than pytest.

From looking at the code there are only two places where it gets disabled:

pytest-cov/src/pytest_cov/plugin.py

Line 134 in 2d2aec1

self._disabled = True

(and line 182). I suggest printing/debugging it from there, but would suspect that somehow you are setting the option.. :)

See also pytest-dev/pytest#4938 - it would be useful here in general, but you can use it as a reference to see where to debug this within pytest.

[ionelmc]

@iamabigstone are you sure you aren't setting PYTEST_ADDOPTS? Cleanup your environment variables.

[iamabigstone]

Somehow cannot repro this issue. closing

[ssbarnea]

I am facing the same annoying warning when I have PYTEST_ADDOPTS='-s --maxfail=10 --no-cov' and when running pytest, I get 7 extra lines of warnings which are useless. If user mentions that he does not want coverage, there is no reason to throw any warnings.

WARNING: Coverage disabled via --no-cov switch!
================================================================================= warnings summary ==================================================================================
/Users/ssbarnea/.pyenv/versions/3.6.8/lib/python3.6/site-packages/pytest_cov/plugin.py:253
  /Users/ssbarnea/.pyenv/versions/3.6.8/lib/python3.6/site-packages/pytest_cov/plugin.py:253: PytestWarning: Coverage disabled via --no-cov switch!
    warnings.warn(pytest.PytestWarning(message))

-- Docs: https://docs.pytest.org/en/latest/warnings.html

Environment should just take precedence over values configured in pytest.ini.

[ssbarnea]

Presence of --no-cov switch should just silence the warning because that is the only way a developer can disable coverage on project having them without altering the codebase. The project itself may have pycov parameters mentioned in its tox.ini or pytest.ini files but once the use defines PYTEST_ADDOPTS, this will take precedence over other options.

I hope that this explains the reasoning behind silencing the warning, as it is just noise. Maybe another approach would be to make a a DEBUG level message, so it will not appear on normal executions.

[reece]

I know that this is closed, but the issue remains. Here's another use case from another voice:

Coverage is enabled by default in pytest.ini with

[pytest]
addopts =
    -rsvx -m 'not extra'
    --doctest-modules --doctest-glob='*.rst' --doctest-glob='*.txt'
    --cov-config=setup.cfg --cov=hgvs

But, while debugging, I want to suppress the coverage, so I add --no-cov:

snafu$ pytest --no-cov -v tests/test_hgvs_alignmentmapper.py
================================================================= test session starts ==================================================================
platform linux -- Python 3.7.5, pytest-5.3.4, py-1.8.1, pluggy-0.13.1 -- /home/reece/projects/biocommons/hgvs/venv/3.7/bin/python3.7
cachedir: .pytest_cache
rootdir: /home/reece/projects/biocommons/hgvs, inifile: pytest.ini
plugins: cov-2.8.1
collected 5 items                                                                                                                                      

tests/test_hgvs_alignmentmapper.py::Test_AlignmentMapper::test_alignmentmapper_AlignmentMapper_HIST3H2A PASSED                                   [ 20%]
tests/test_hgvs_alignmentmapper.py::Test_AlignmentMapper::test_alignmentmapper_AlignmentMapper_LCE2B PASSED                                      [ 40%]
tests/test_hgvs_alignmentmapper.py::Test_AlignmentMapper::test_alignmentmapper_AlignmentMapper_LCE3C PASSED                                      [ 60%]
tests/test_hgvs_alignmentmapper.py::Test_AlignmentMapper::test_alignmentmapper_AlignmentMapper_PTH2 PASSED                                       [ 80%]
tests/test_hgvs_alignmentmapper.py::Test_AlignmentMapper::test_alignmentmapper_failures PASSED                                                   [100%]
WARNING: Coverage disabled via --no-cov switch!

=================================================================== warnings summary ===================================================================
venv/3.7/lib/python3.7/site-packages/pytest_cov/plugin.py:278
  /home/reece/projects/biocommons/hgvs/venv/3.7/lib/python3.7/site-packages/pytest_cov/plugin.py:278: PytestWarning: Coverage disabled via --no-cov switch!
    warnings.warn(pytest.PytestWarning(message))

-- Docs: https://docs.pytest.org/en/latest/warnings.html
============================================================ 5 passed, 1 warning in 10.64s =============================================================


It seems to me that I'm being warned about something that I explicitly asked for and that isn't an error condition.

This is pytest 5.3.5, cov-2.8.1

[ssbarnea]

I am glad I am not alone on this, as I didnt want to be the boy that cried wolf...;)

IMHO, lots of issues could be fixed if we assure that installing coverage module does not activate any kind of coverage by default. It should need another option to activate it (config, cli, env).

Just today I had to do a workaround on a project where I was running “pytest —collect-only” before running the test and coverage was producing multiple files and causing some problems on CI.

[blueyed]

@ssbarnea

IMHO, lots of issues could be fixed if we assure that installing coverage module does not activate any kind of coverage by default. It should need another option to activate it (config, cli, env).

You are using --cov, aren't you? :)
What I mean is that I'd rather also use this explicitely (CI, tox env), and not have it in addopts by default for example.

Apart from that I agree that --no-cov should not produce a warning probably. The issue was about something else though (the warning without the flag (seemingly). Please consider create a new one, or a PR right away I would suggest.

[ionelmc]

The problem is that people are using it to configure the sources. It's like a workaround: activate coverage just to configure sources, then disable it for some one-off run.

The previous maintainer (Marc?) has an idea or wip branch to remove the dual --cov vs --cov=source use (and also avoid the --cov test_stuff.py problem in the process) but back then I considered it inopportune to change (given the many worse problems the plugin had back then). I have to check a bit more in the history but I think we could reconsider that idea and implement some sort of --cov-source or similar to let people configure the source without activating. On the other hand the source can be configured in the coveragerc in theory - why aren't you using that?

Anyway, I'll take a look again at #304 - looks like I forgot about it or closed it by accident. Does it solve most of the complaints?

[reece]

@ionelmc: What do you mean by "The problem is that people are using it to configure the sources." ?

If you're saying that I've not configured this the way the pytest developers intended, I'm not offended or surprised: I do find configuring pytest + plugins to be confusing!

What should I have done instead?

[blueyed]

@reece IMHO the problem is that you are enabling it by default. It's really useful on CI anyway (where it can be combined etc), but for local runs a "coverage" tox factor might be useful already also.

I agree though that it should be not really a warning though also.

[vanschelven]

IMHO the problem is that you are enabling it by default.

But the you in that sentence is not necessarily the same you who's suffering from the problem; i.e. you might be working on a project where you have no control over tox.ini, but still want to locally turn off coverage reports.

[webknjaz]

One workaround would be to explicitly instruct pytest to ignore this specific warning:

# pytest.ini:
filterwarnings =
    error
    ignore:Coverage disabled via --no-cov switch!:pytest.PytestWarning:pytest_cov.plugin

This may be not ideal as you'd probably want it to explode in CI but at least it's less annoying. Besides, if you report coverage to third-parties, they'll probably alert you if the coverage suddenly drops anyway...