MMF-1134 (pull requests are first class citizens in SonarCloud), teams can activate PR decoration on their repository. For that, they need (amongst other things) to provide a GitHub token that they will set through the administration web page of the project. This has the following drawbacks:
- The team will have to create a "technical" GitHub user for which they will generate tokens
- This will be the user who will appear as the one commenting on the PRs
- The team will have to set those tokens on every project
- If the tokens get revoked (for instance because one was compromised), then the team has to regenerate a new one and set it again
Using a GitHub application, SonarCloud can request access to the user's repositories, and therefore remove all of those limitations at once.
We want SonarCloud to be able to decorate pull request:
- with its own identity
- without requiring any user token
Note: the following description still has to be confirmed by an innovation sprint. It is here as a guidance for that sprint.
2 solutions were currently discussed, that should be investigated during an innovation sprint to validate the concept:
- Link a SonarCloud organization to remote a GitHub org and/or a VSTS org/account (and tomorrow to a Bitbucket team and/or GitLab organization)
- Probably the best solution
- Expected UX:
- As an org admin on GitHub, when I am on SonarCloud, I have a way to "link" my SonarCloud org to the GitHub one. This action probably brings me to a GitHub screen asking to install the SonarCloud app on my GitHub org, granting all the required rights to do what we need on PR.
- The consequence being that as soon as a PR is analyzed on a project that belong to this SonarCloud org, based on the repo key passed by the scanners, SonarCloud CE can automatically figure out that it has the relevant access rights to update the status of the PR and comment it w/o requiring any user token
- The benefit of this solution is that the link is done once at org level.
- Link a SonarCloud project to a remote repository
- Less ideal solution
- Expected UX:
- As a project admin, when I am on SonarCloud, I have a way to "link" my project to a remote repository, probably through a dedicated page on which I can browse remote repositories that I have access to. This action probably brings me to a GitHub screen asking me to grant access to this repository.
- The drawback of this solution is that the link needs to be done manually on every project.
- This solution should be investigated only if the first one is a dead end.
Innovation sprint proved that it is possible to perform PR decoration through a GitHub SonarCloud Application.
Knowledge gathered during the innovation sprint is available in
- confluence page
- sonar-enterprise branch feature/sl/POC_Github_application
- branch feature/sl/github_app_support of SonarSource's fork of the github-api
We will go with a solution close to solution 2: we will store the link between a Github Orgnization and an Installation ID (see below).
A GitHub Application has a unique ID, requests permissions a user must accept to give when installing, can register to Webhook events, etc.
A Github Application must be installed by an organization's owner. This creates a so called "GitHub Application Installation" ("Installation" for short) which has a unique ID.
A Github application can interract with Github through a subset of the API V3 (work in progress on Github's side) with its own identity.
To do so, though, it must use an access token for a specific Installation, which has a 10 minutes lifespan.
As a SonarSource user (application are linked to a user, not an organization), create a new Github Application:
- Name should be "SonarCloud"
- Description to be defined
- define a homepage URL (content and URL TO BE DEFINED)
- no authorization callback URL
- no setup URL
- define a URL to receive webhooks on (eg. https://www.sonarcloud.io/integration/github/app_webhook)
- define a LOGO (TO BE DEFINED)
- when going public, request the application to be added to Github's marketplace
- request the following permissions
- Commit statuses: Read & write
- Pull requests: Read & write
- define a webhook secret to secure webhook calls received from Github (see security below)
- register to not specific events to get Webhook notifications
- the only notifications we need are always sent to an application (see Storing the GitHub Repository to Installation ID link)
We will leverage the webhooks sent by Github to the SonarCloud Application to get this link and store it on SonarCloud's side.
The following operations will create events which will be notified by webhook:
- Organization owner installs the application
- Organization owner changes the repositories accessible to the application
- a new repository is created and "All Repositories" in the organization are accessible to the Application
- a repository is deleted and "All Repositories" in the organization are accessible to the Application
- a repository is deleted which was explictly made accessible to the Application
When receiving these events' webhooks, we populate a DB table containing (eg. named GITHUB_APPLICATION_INSTALLS):
- the Github organization's slug
- the ID of the Installation
Currently, PR decoration is performed using a Github user's oAuth token provided through SonarCloud's UI.
Support for this feature is kept as a fallback if the SonarCloud Github Application is not installed on the project's Github Repository.
To perform PR decoration, the initial interaction with Github will slighly change. Rather than take the OAuth token from the property, we will:
- look up table GITHUB_APPLICATION_INSTALLS whether we know of a Installation ID for the project's Github organization.
- if none exists, attempt PR decoration with OAuth token if there is any
- otherwise create a JWT token using the Github Application's private key
- such JWT token can have a lifespan of 1 hour, we may not create a new one for each PR decoration (but that would require storing it in DB, etc.)
- use this JWT to create an Installation Access token for the Installation ID we found
- check whether the current repository is accessible to the Installation (eg. by attempting to list the project's Pull Requests)
- if so, perform PR decoration with this token, otherwise fallback to OAuth token if there is any
To interact with Github as a Github Application, one must create a JWT token signed with the Application's private key. The private key is stored in PEM file and can be generated by the Application's owner on the Application's page.
The path to this PEM file will be specified through a property.
We will have to securely make this PEM file available on SonarCloud App Nodes servers.
To receive webhooks, we are exposing a endpoint to the wild Internet.
SonarCloud/SonarQube authentication should not apply to this endpoint but it will be secured with the following:
- only requests from Github public IPs will be accepted
- content of webhook's payload will be validated using the Application's Webhook Secret
The Application's Webhook Secret will have to be specified through a property.
SQ will hash the webhook's payload using the secret and compare the hash with the value of header X-Hub-Signature.
Notes for later:
- When we detect (on CE side) that we are processing a PR but that the org (or project - depending on the solution) is not linked to any remote service, we should provide that feedback to the user and help her/him connect his org (or project) to the remote service
There is very few differences between decorating with a user's OAuth token and with an Installation access token:
- github-api library has to be slightly updated to not attempt to retrieve the current user when using a Installation access token
- we need to know the application's id
- TODO where should we retrieve it from? SQ property?
- the application's name is what replaces the user's login in comments and we use it to recognize "our" comments from other users while decorating a PR
- TODO where should we retrieve the application's name from? from a property in SQ? from an Github API call? If so, when should we do it?