We believe that tests are important and are detecting more relevant issues in tests files so that developers can improve the tests they are writing and fix issues that can impact the code which goes in production.
We want to make sure that users perfectly understand which issues apply to the main code and to the test code.
As a developer
- I expect by default, my test code to be analyzed in SonarQube.
- I expect issues on test code to contribute to the default Quality Gate.
- If I'm not really ready to care about test code, I expect the transition to be smooth.
- I want to easily distinguish main and test issues in case I want to prioritize them. I may want to start fixing bugs in my main code because fixing my main code might impact my tests.
For example, on the Issues page, I expect to be able to filter issues related to main or test files.
Issues on tests already and should keep contributing to the main ratings on New Code the same way they currently do. They will naturally be taken into account in the default Quality Gate.
It will allow the transition to be smooth for those who care about test code quality:
- For users who where already analyzing tests, nothing will change.
- For users who will start analyzing tests, since the criteria of the default Quality Gate are on New Code, the releasability of projects that follow Clean as you code should not be impacted much. Still, since we'll report more and more issues on test code, users who use overall criteria in their Quality Gate definition can expect an impact on the Quality Gate status of their projects.
- Also, as a first step, we'll take care to execute rules that bring high value to the user, rules which are specific to tests. This should avoid reporting a lot of new very generic issues which could be to be perceived as noise, even for users who have added overall measures in their Quality Gate.
As a developer, I mainly focus on the New Code and I expect to fix all the issues raised on this Code for my PR to me merged or in the branch that I want to release.
- At a high level, I don’t need to distinguish quality on test code from quality on main code: I want to fix all these issues, whenever they are on main or test code.
- Still, when I'm looking at the issues, I want to be able to make the distinction. For example, I may be interested to first look issues on main code, so that if I have to change my main code, I'll have a look at issues on tests when I've adjusted the test code accordingly.
- When navigating in the code (Measures and Code spaces), the distinction between main and test code is done naturally while drilling down into the files hierarchy. Still, an indication would help me to quickly understand when I am looking at main or test code.
The scope displayed in the rules description (main, test or both) corresponds to the theoretical scope to which the rule should apply but it's not nowadays taken into account during rules execution.
Until it becomes the case, we'll remove this information which appears to be confusing.
We add a new facet in the left sidebar to allow users to filter issues accordingly to their source (main or test)