Uploaded image for project: 'Marketplace Extension for Azure DevOps'
  1. Marketplace Extension for Azure DevOps
  2. VSTS-179

Coverage and unit tests are no longer automatically imported

    Details

      Description

      Looks like Microsoft did some changes on the vstest task v2 which breaks the auto-import feature for coverage and tests.

      Interesting discussions from the community:

      And a hint from Daniel Svensson: look at https://github.com/Microsoft/azure-pipelines-tasks/issues/7975

       

      Related ticket in Scanner for MSBuild for a partial workaround:

      https://github.com/SonarSource/sonar-scanner-msbuild/issues/724

       

      Update: depending on how the user configures the VSTest task, it may generate a new runsettings file to pass the configuration to the test runner. If this happens, then the task will also change the location of test output -> our analyze task won't find the test results -> no coverage uploaded.

      The suggestion from Microsoft is to use the REST APIs to check for test artefacts and re-download them to the server. This is what we do for the legacy XAML builds, but it's ugly in practice. For legacy XAML builds, the test artefacts need to processed on the server before they are available via the REST APIs, so if we query the APIs too soon we might be a result indicating that there are no test artefacts. That makes it impossible to tell the difference between the case where there really are no test artefacts, and the case where there are artefacts but they haven't been processed yet. For legacy XAML builds we went for the ugly solution of polling the server repeatedly with a timeout.

      In cases where there are no test results this unnecessarily delays the build. I think we also had the added complexity of having to introduce a timeout parameter users could set to increase the timeout if necessary to make sure the test artefacts were correctly discovered when they did exist.

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                duncan.pocklington Duncan Pocklington
                Reporter:
                fabrice.bellingard Fabrice Bellingard
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Due:
                  Created:
                  Updated:
                  Resolved: