Plugin support for dynamic context

[sjustas]

This PR allows writing simple dynamic context plugins.

class TestMethodPlugin(coverage.CoveragePlugin):

    def dynamic_context(self, frame):
        if frame.f_code.co_name.startswith("test_"):
            return coverage.context.qualname_from_frame(frame)
        return None

def coverage_init(reg, options):
    reg.add_dynamic_context(TestMethodPlugin())

I kept the dynamic_context = test_function setting for backwards compatibility.

[codecov-io]

Codecov Report

Merging #783 into master will decrease coverage by 0.06%.
The diff coverage is 90%.

@@            Coverage Diff             @@
##           master     #783      +/-   ##
==========================================
- Coverage   89.98%   89.91%   -0.07%     
==========================================
  Files          78       78              
  Lines       10830    10937     +107     
  Branches     1118     1128      +10     
==========================================
+ Hits         9745     9834      +89     
- Misses        959      977      +18     
  Partials      126      126

Impacted Files	Coverage Δ
tests/coveragetest.py	100% <100%> (ø)	⬆️
coverage/plugin_support.py	52.86% <100%> (+0.91%)	⬆️
coverage/plugin.py	84.84% <50%> (-1.09%)	⬇️
coverage/control.py	86.44% <87.5%> (-1.8%)	⬇️
tests/test_plugins.py	96.76% <90.69%> (-1.66%)	⬇️
coverage/collector.py	56.5% <0%> (-0.5%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6309cd4...3865057. Read the comment docs.

[nedbat]

@sjustas Thanks for doing this. I've been meaning to make time to look at it, but I have a conference coming up. It may take me a few more weeks, sorry.

[sjustas]

@nedbat, no worries, I'll be looking forward to your feedback.

[nedbat]

@sjustas This is very comprehensive, thanks. I appreciate the work you put into the tests, which are significant.

Do you have a use case for chaining the context switchers? It would be easier to just have one in effect at a time, but maybe you had a need to have more than one?

[sjustas]

This could be useful for supporting multiple different TestSuite implementations. Python is pretty flexible there, even bundled doctest has DocTestFinder which generates DocTestCase instances from .txt files it finds (like "Readme.txt"), so someone might want write a plugin that sets a context for these tests.

Another example could be RMLRenderingTestCase. It would probably handle contexts in similar way to DocTestCase, but in theory people can make all sorts of test suites and test cases. It would be nice if the context names would be meaningful.

So I thought it would be nice if developers could implement context plugins that work correctly with each library, and users of these libraries (say unittest + doctest + z3c.rml) could just list all the plugins in configuration.

I think another good way to approach this is to have an option for test runners themselves to set correct contexts for each test case they run, hence the PR #782 🙂

[nedbat]

Thanks! I'll be doing some clean up after the merge.