Prototype pytest plugin for test discovery/execution.

[karthiknadig]

• Ensure that any additional plugins that modify id like hypothesis are handled.
• Ensure that pytest is be launched using pytest binary (pytest.exe or /bin/pytest) or as module (python -m pytest).
• Ensure that result is printed to stdout if server is not available (this should help with debugging)

This implementation should be no different than any other pytest plugin, this way we don't interfere with how pytest detects tests, or id generation, test selection, or execution. While debugging tests this plugin will be added to the ignore list in the debug configuration.

[eleanorjboyd]

@karthiknadig can you confirm these second two bullet points are set with the current status of the rewrite? Additionally I will put it on the todo list to test with hypothesis

[karthiknadig]

@eleanorjboyd yes. I just verified those.

[eleanorjboyd]

just checked and the hypothesis plugin works with the new rewrite. steps to check:

1. install hypothesis lib python -m pip install hypothesis
2. discover and run tests with this test file and it works

file name: test_ex.py

from hypothesis import given, strategies as st


@given(st.integers(), st.integers())
def test_addition_is_commutative(x, y):
    assert x + y == y + x

[andreamah]

I tried to get this to work, but the test seems to be running forever. Am I doing something wrong?

[rzhao271]

The test worked on my end, though I had to restart while setting up because I tried discovering the tests, only to realize I forgot to create an environment, so then I tried creating an environment, which put the editor in a condition where it kept failing to discover tests after. If I can repro such a setup, I'll file an issue for it.

[eleanorjboyd]

@andreamah, could you file a bug on this with logs?

@rzhao271, thank you definitely file a bug if you see it again. Thanks!