IssueRaiser

Enable multiple test suites and test configurations in the Test UI

<!-- Please search existing issues to avoid creating duplicates. --> <!-- Describe the feature you'd like. --> ## Problem Statements * P1. Currently VS Code for Python only allows a single test configuration for each test framework. For example, using PyTest, I can apply certain arguments to run whenever PyTest is run through the UI, but if I setup that configuration to exclude certain tests (such as those annotated with "slow"), I have to manually update the arguments in the options window to switch to another configuration. * P2. There is also no way to setup separate groups of tests and show them in the Test sidebar window as those groups. For example, rather than having a module-based hierarchy, it would be great to have the ability to have them grouped by arbitrary groups of modules/classes/methods and/or based on annotations. Currently one cannot have the tree show different groups of test, such as those annotated as "slow" or "sanity". * P3. In addition, any tests that are excluded by the current test configuration cannot be run even when they are directly selected to run as an individual test, because the current configuration causes them to be excluded by the test framework. So that one test will be detected, but then excluded from running. So if I normally want to exclude "slow" tests, that means I cannot even click directly on a slow test and run it directly in the UI. ## Current Elements in the UI: * Existing layout in "Test" sidebar window: ![image](https:

Posted by LightCC on May 30, 2020, 5:54 PM

Tags

Related issues

Raise real-world problems, find people facing the same issue, and discuss solutions.3,500 issues0 comments

ux (Other)

Enable multiple test suites and test configurations in the Test UI

LightCCMay 30, 2020, 5:54 PM

<!-- Please search existing issues to avoid creating duplicates. --> <!-- Describe the feature you'd like. --> ## Problem Statements * P1. Currently VS Code for Python only allows a single test configuration for each test framework. For example, using PyTest, I can apply certain arguments to run whenever PyTest is run through the UI, but if I setup that configuration to exclude certain tests (such as those annotated with "slow"), I have to manually update the arguments in the options window to switch to another configuration. * P2. There is also no way to setup separate groups of tests and show them in the Test sidebar window as those groups. For example, rather than having a module-based hierarchy, it would be great to have the ability to have them grouped by arbitrary groups of modules/classes/methods and/or based on annotations. Currently one cannot have the tree show different groups of test, such as those annotated as "slow" or "sanity". * P3. In addition, any tests that are excluded by the current test configuration cannot be run even when they are directly selected to run as an individual test, because the current configuration causes them to be excluded by the test framework. So that one test will be detected, but then excluded from running. So if I normally want to exclude "slow" tests, that means I cannot even click directly on a slow test and run it directly in the UI. ## Current Elements in the UI: * Existing layout in "Test" sidebar window: ![image](https:

Comments (0)

No comments yet. Start the discussion.

Comment on this issue