Uploaded image for project: 'SonarQube'
  1. SonarQube
  2. SONAR-12665

Fix and Automate sonar-enterprise release

    Details

    • Type: Task
    • Status: To Do
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Edition:
      Community

      Description

      WHY

      The release process of sonar-enterprise has recurrent issues with producing the released artifacts and contains many manuals steps which makes the whole procedure a risky and painful event.

      On top of that, the current procedure implies a freeze of push to the master branch which affects the operation of the whole team and we would rather avoid.

      WHAT

      4 digit version everywhere!

      The first and mandatory issue to fix with the release process is that sonar-enterprise is probably the only project which breaks the principle of releasing artifacts which were tested and dogfooded.

      For historical reasons (which are quite likely no more valid today), released version of SonarQube are using 2 or 3 digits version number (eg. 8.0, 7.9.1).

      As a consequence, we make a specific commit to master to change the version, build that commit (and create released artifacts from there) and right after that push another commit to get back to normal.

      Building the first commit more than once is bad, because we produce multiple artifacts with the same name, and it happens!

      Impacts

      Display of "functional" version in SQ

      Another reason we don't use 4 digits version: the current version of SonarQube is available from SQ itself at runtime. This is an important information for users and for support.

      Moving to 4 digit version, we would add the build number everywhere.

      user should still have a way to tell, from her own SonarQube alone, that this 4 digit version is 8.0 or 8.1 or 7.9.2.

      SQ version is displayed in several places in the UI:

      • the footer

        the current display is ok and we need it on Next. However, it doesn't bring value to the user so it could be made less visible (tooltip?)
      • multiple locations in system info page
        it's ok to display the 4 digit version there
      • when new version is available ( note that this means that older versions of SQ will be also be impacted by the change when 8.1 is released with a 4 digit version)

         the version comes from the UpdateCenter properties file where we can still write "8.1" and specify a download URL with an artifact with a 4 digit version.
      • ...

      SQ version is available from several WebServices:

      • api/system/info
      • api/system/status
      • ...

      Zip file names

      User will now download an artifact named sonarqube-8.1.0.xxxx.zip when downloading the ZIP for the developer edition of Sonarqube 8.1. They used to download  sonarqube-8.1.zip. Also, zip currently unpacks by default to a directory named sonarqube-8.1 and this will change too.

      note 1: the file name situation already exists with language plugins, for example

      note 2: there will be only one public release for a given "functional" version at any point in time

       Should we worry about it from user point of view? product marketing said it doesn't matter

       Is there an impact on the SonarQube and SonarSource websites? marketing said there is no automatic thing in the website => no impact

      Is there an impact on scripts and tools involved in the release process?

       Is there an impact on Docker images?

       It won't be possible anymore to "guess" download URLs for SQ zip files. Do we care? we don't, very marginal use case

      Identifying released versions of SQ

       How will support know that 8.1.0.xxxx in the system info they receive is the public and official 8.1 version?

       How will we identify in Artifactory, Burgr and other tools that 8.1.0.xxxx is SonarQube public and official 8.1 version?

      Maven central artifacts

      SonarQube artifacts (which includes sonar-plugin-api) published to Maven Central will have 4 digits now, similarly to scanners:

       is it a problem? product said it's not a problem

      Documentation

      Version of SonarQube is displayed in the switcher, in the URL and in many places in the documentation

       no impact in the switcher nor in the URL. shipping doc from 4 digits artifacts is actually a feature since we are able to deploy fixes to the doc without releasing SonarQube

       version in the doc itself is hardcoded and for the reason above, it also not a problem

      SonarLint

      SonarLint code include tests here and there on the version of SonarQube.

       no problem expected as SonarLint works with SonarCloud, Next and Peach who run 4 digit versions SonarQube instances

      License verification

      Seems like only edition and server id are checked

      Telemetry

      Current version and installation version of SonarQube are sent in telemetry data

       4 digits version are already sent => no problem

      Update Center and plugins

      Update Center is dealing with "functional" versions (ie. which do not contain the build number) for compatibility matrix with SonarQube.

      Update Center in SonarQube and loading of plugins (check of compatibility) should not be impacted as any non-release build of SonarQube is correctly displaying Update Center content and checking compatibility.

      For the same reason, Update Center library and Mojo should work correctly.

      Project dump

      Project dump feature works only from a version of SonarQube to the same version.

      The verification of the version should not be impacted as it already works with non-release versions of SQ.

      Plugin updates

      Each new release of SonarQube must bundle the most recent version of the analysers, the scm plugins, ...

      This is currently handled with a ticket (eg. SONAR-12252). This ticket is usually taken care at the last minutes of the release.

      There are several issues with this:

      1. plugins are updated at the last minute, the release is done right after that and SonarQube running with these plugins has not been dogfooded
      2. it is a pain (because manual) to find out which version is the latest one of each plugin
      3. it is a pain (because manual) to make sure all plugins have been updated

      This could be fixed by automating the process of update plugins and make the update happen as the plugins are released.

      Related (ie. nice-to-have)

      Documentation release

      Online documentation for SonarQube must be updated with each release.

      Updating the documentation to the latest version is currently a manual operation. It's currently handled by PMs as part of the functional release. 

      This operation could benefit from being automated (easier and less prone to errors).

      Docker images release

      Releasing the SonarQube Docker images implies several manual operations and we are not even sure today of how to release a version but the first one of a major version of SonarQube.

      HOW

      Update release procedure

      TO CONFIRM: SonarQube should now be able to use the one-click release feature used to release scanners and plugins (see https://xtranet.sonarsource.com/display/DEV/Release+Procedures).

      Change to SQ Gradle build

      • gradle.properties now always contain a version without -SNAPSHOT
      • when no buildNumber is provided, artifacts are all build with a version ending with -SNAPSHOT
        (this means reversing the logic in build.gradle of sonar-plugin-api)

      Testing the new release procedure

      Version 8.0.1 of SonarQube will be released with the new procedure but will never be made public.

      This will leave out testing of publication of Maven artifacts and the PM part of the release process.

      This is ok, they will be tested with the release of SonarQube 8.1 which comes mid-December.

      Plugin update bot

      Mimicking what was done on SonarCloud side (see SC-1176), we want a bot to open pull requests on the sonar-enterprise repository when any of the bundled plugins get a new release.

      Usage of PR has several benefits:

      • a developer doing the upgrade will anyway have to open one, it might as well be done automatically
      • QA will run on the PR as soon as it is opened. We will automatically know if the upgrade fails the IT or has other impact.
      • if QA is green, it might take nothing more than a click in Github UI to be done with it

      However:

      •  who's going to take care of these PRs
      •  shall we always have a single PR by plugin?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                sebastien.lesaint Sebastien Lesaint
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: