VSCode-Python/unittest module invoking wrong TestCase?

[alabasterfox]

Environment data

• VS Code version: 1.40.2
• Extension version (available under the Extensions sidebar): 2020.2.64397 (21 February 2020)
• OS and version: Linux x64 3.10.0-1062.el7.x86_64
• Python version (& distribution if applicable, e.g. Anaconda): Python 3.6
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): virtualenv (also tested N/A)
• Relevant/affected Python packages and their versions: unittest (deploys with python)

Expected behaviour

As a User, in the Test Panel of VSCode, with the 3 TestCases USER_STORY_1_8_1, USER_STORY_1_8_10, and USER_STORY_1_8_11 present, when I click the run or debug icon associated with a TestCase named USER_STORY_1_8_1, TestCase USER_STORY_1_8_1 is run/debugged.

Actual behaviour

As a User, in the Test Panel of VSCode, with the 3 TestCases USER_STORY_1_8_1, USER_STORY_1_8_10, and USER_STORY_1_8_11 present , when I click the run or debug icon associated with a TestCase named USER_STORY_1_8_1, TestCase USER_STORY_1_8_11 is run/debugged. If USER_STORY_1_8_11 is unavailable (e.g. commented out), USER_STORY_1_8_10 is run/debugged. And if neither is available, only then is USER_STORY_1_8_1 run/debugged.

Steps to reproduce:

1. Create the a new VSCode project with a file named test.py and copy and paste in the following simple code:

import unittest

class USER_STORY_1_8_1(unittest.TestCase):

    def setUp(self):
        print('USER_STORY_1_8_1:setUp')

    def tearDown(self):
        print('USER_STORY_1_8_1:tearDown')

    def test_run(self):
        print('USER_STORY_1_8_1:test_run')

class USER_STORY_1_8_10(unittest.TestCase):

    def setUp(self):
        print('USER_STORY_1_8_10:setUp')

    def tearDown(self):
        print('USER_STORY_1_8_10:tearDown')

    def test_run(self):
        print('USER_STORY_1_8_10:test_run')

class USER_STORY_1_8_11(unittest.TestCase):

    def setUp(self):
        print('USER_STORY_1_8_11:setUp')

    def tearDown(self):
        print('USER_STORY_1_8_11:tearDown')

    def test_run(self):
        print('USER_STORY_1_8_11:test_run')

test.py

2. Configure VSCode to use a python interpreter for execution and the unittest framework for testing.
   When configuring the unittest framework, configure auto-discover to search for files with the pattern test*.py

3. On the left sidebar click the Chemistry flask icon to open the Test Panel of VSCode. The 3 unit tests will be auto-discovered and populated in the list under test.py

4. Go to the TestCase named USER_STORY_1_8_1 and click the run icon or debug icon associated with it to attempt to run it or debug it

Issue is encountered: TestCase USER_STORY_1_8_11 will be run or debugged instead of the target USER_STORY_1_8_1 TestCase

I have a corresponding stackoverflow question open here: https://stackoverflow.com/questions/61048113/python-unittest-module-invoking-wrong-class where another user confirmed seeing the same behavior and proposed opening an Issue Report

[karrtikr]

I can confirm the bug, thanks! As quoted in stack overflow, Please consider using a different test framework as a workaround. I've checked the same code but running it with the pytest framework and all tests run without any issues.

[karthiknadig]

This is addressed by #16769 Clicking run on USER_STORY_1_8_1 runs just that.