Uploaded image for project: 'SonarQube'
  1. SonarQube
  2. SONAR-7736

Fix behavior of executable_lines + IT and overall coverage

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Coverage & Tests, Scanner
    • Labels:
      None

      Description

      In 5.5 we have introduced a new metric executable_lines that is used to default UT line coverage to zero when no data is provided by sensors. The problem is that as soon as you have a few files covered by ITs the global metrics are not consistant.

      Example: let say we have a project with 2 files. FileA and FileB. Each file has 10 executable lines correctly reported by language plugin.

      1) When no coverage is reported at all, current behavior is to save 0 UT hits on each executable line. We don't save IT hits to avoid the UI to start displaying ITs metrics/coverage when most projects have no IT. OK.

      2) Now let say Jacoco reports 5/10 UT covered lines on FileA (FileB is uncovered). The platform will save 0 hits on FileB. UT coverage of is 50% for FileA, 0% for FileB, 25% for the project. Still no IT / Overall coverage displayed in UI. OK.

      3) Now use the same JaCoCo report as IT coverage (no UT). IT coverage is 50% for FileA, unset for FileB, so 50% for the project => KO inconsistent with case 2.

      4) Now use the same report twice, once for UT and once for ITs:

      • FileA: UT=50% IT=50% Overall=50%
      • FileB: UT=0% IT=unset Overall=unset
      • Project: UT=25% IT=50% Overall=50%

      We have two opposite requirements:

      • in a perfect world we would like to compute UT and IT coverage of all files.
      • but we don't want to show "red bar" on all files because of missing ITs since this is too aggressive.

      In multi language project (Java + JS) it is very common to have UT for Java and JS, but ITs only for Java. Showing 0 IT coverage on all JS file is a bit strong. On the other side, not counting executable lines of JavaScript files in ITs will make project measures inconsistant (as well as overall).

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                julien.henry Julien Henry
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: