Uploaded image for project: 'Product Roadmaps'
  1. Product Roadmaps
  2. MMF-969

Minimal support for Short-Lived Branches Analysis in SonarQube

    XMLWordPrintable

    Details

      Description

      The Problem

      A short-lived branch is a branch whose distance to the main branch is 99% of the time short in term of number of changes and which is supposed to be merged into the master as quick as possible (matter of days or weeks). Obviously if any quality issue is created in a short-lived branch the goal is to get this piece of information before doing the merge. For the time-being there are 3 ways to get this feedback and none of them is really convenient:

      • SonarLint : that's convenient to get a local feedback but SonarLint is really about getting a feedback on-the-fly while coding and not so much to get the big picture on all the potential issues part of a short-lived branch
      • SonarQube preview mode: this mode is currently deprecated and is not really integrated into SonarQube
      • GitHub PR Analysis: very limited to GitHub and again not integrated into SonarQube (no way to benefit from the SonarQube Issue brower, to tag issue, ...)

      The Idea

      Having in SonarQube and for a dedicated project, all the pending short-lived branches. For each short-lived branches we just need :

      • The number of new bugs, code smells, vulnerabilities
      • The ability to browse those issues in the context of the updated source files
      • The ability to update each issue: FP, change severity, comment, ...
      • Optionally The ability to associate a link to the short-lived branch to browse the changes directly in the SCM UI.
      • Being notified with help of a web hook when a short-lived branch is created or updated
      • Tracking "at best" new issues in the short-lived branch by using the main branch and by making the assumption that the distance between the two branches is short enough to make the issue tracking algorithm remaining relevant.

      Out of scope :

      • Computing some measures on a short-lived branch (except the number of bugs, code smells, vulnerabilities and files) -> and so computing a Quality Gate status
      • Once a short-lived branch is merged, any issue coming from this short-lived branch will be considered as a new issue in the main branch and so any update (comment, resolution status, ...) done in the short-lived branch will be lost.
      • Having SonarQube automatically updating the status of the relating branch in a code review tool like GitHub, Bitbucket, GerritForge, ...
      • Using different settings or quality profiles to analyze my short-lived branch
      • Linking my SonarLint to a short-lived branch

      To be noted that this mechanism could also be used for pre-commit check even if that would be a kind of hack. No threshold mechanism to limit the number of updated source files part of a short-lived branch.

      A new /api/branches WS should allow to search for branches. There is no mandatory filtering criteria but a project key and/or a partial branch id can be provided.

      If we would also like to provide a mechanism to propagate all issues changes from short-lived branches to long-lived branches the following basic algorithm should work :

      • In case a new issue appears in a long-lived branch
      • Is there any issue existing on a short-lived branch on the same rule, on the same file and with the same line checksum ?
        • No -> do nothing
        • Yes -> match and copy the issue from the short-lived branch to retrieve all comments, status, assignee, ...

      The Solution

      The current property sonar.branch must be deprecated and its current behavior should not evolve.

      A new analysis property must be used to activate the real support for branches :

      • sonar.branch.name: unique identifier for this branch in this project

      When this analysis property is used we do expect to see a new or updated entry in a a project branches list. This page or dropdown list should contain a scrollable list of analyzed short-lived branches and for each branch the following information should be available :

      • id of the branch
      • time of the last analysis (most recent analysis should be displayed first)
      • number of new bugs
      • number of new code smells
      • number of new vulnerabilities
      • number of updated source files
      • Link to the SCM UI if any

      Moreover a search field a the top of this "Branches" page should allow to quickly filter the list of displayed branches.

      Clicking on a row should provide the ability to browse the new issues part of this branch and no more.
      After X days (configurable with a setting parameter and by default 100 days) without analysis, those short-lived branches must be deleted (and all the relating content in the back-end). It must also be possible to manually trigger the deletion of a branch from the list. When doing such analysis of short-lived branches we do expect to analyze only the updated source files as it's currently done with GitHub PR Analysis (kind of incremental mode). Obviously if a project is deleted all its relating branches must also be deleted.

      A branch should not be searchable from the global search engine and should not be displayed in the Projects page. Moreover, the issues part of a branch should not be accessible outside of this branch (so not in the global or project Issues pages for instance).

      About the Design

      Scanner

      The goal would be to not depend upon any SCM and so to guess what has changed on our side with the following basic algorithm:

      • First we start by excluding all the source files having the same checksum on both sides (what is already done with the preview mode to determine which source file must be analyzed)
      • Then for each updated source file and if SCM blame information is available, we check on which side the most recent update has been done. If the most recent update has been done in the short-lived branch or if the last update has been done after the first analysis of this branch, we do consider that this source file has been updated part of this branch.

      Back-end

      Each branch is going to be considered as a special project within SonarQube. By doing that we automatically inherit from the support of measures and issues. In the projects table we need to create the following new columns:

      • BRANCH -> the id the branch (see sonar.branch.name). This column is expected to be fed at project level and for all the underlying components.
      • BRANCH_PROJECT_UUID -> the UUID of the project this branch belongs to. This column is expected to be fed only at project level.

      Moreover we also expect in the ES 'issues' and 'components' indexes to have a boolean "IN_BRANCH" field for two reasons :

      • To filter out all the components part of a branch in the global search form
      • To filter out the issues part of a branch in the global/project search form

      As this is already the case, the key of a branch should be an aggregation of sonar.projectKey and sonar.branch.name.
      When doing anything on a branch the permissions to be used must be the one of the BRANCH_PROJECT_UUID. As a consequence trying to update the permissions of a branch with help of api/permissions should not be possible.

      Issue Tracking

      The issue tracking mechanism must be a bit more elaborated for short-lived branches because :

      • We must first detect the issues which already exist in the long-lived branch
      • And then for the new issues we must match the ones that already existed during the last analysis of this short-lived branch

      Others

      • No branch analysis should be authorized if the relating project doesn't exist
      • In the "Background Tasks" pages, we do expect to retrieve for each row relating to the computation of a short-lived branch analysis report :
        • The name of the project
        • The icon of short-lived branches
        • The name of the short-lived branch

      Current design iteration

      SonarQube_Branches_Prototype.zip

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              fabrice.bellingard Fabrice Bellingard
              Reporter:
              freddy.mallet Freddy Mallet (Inactive)
              Votes:
              3 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: