Validation feedback is an essential element in Human Computer Interaction, especially when integrating with other tools. Users need to know what they just configured worked in order to proceed confidently with their next goal.
During our latest usability testing study we identified that users expect feedback regarding the validity of their configuration.
We also saw a lot of community posts about users struggling to configure SonarQube integration with their ALM. Most of the time they have no clue what they did wrong and reach out to us for help.
Therefore we should improve the ALM integration task by checking the validity of the integration configuration as early as possible.
ALM Integration configuration domain covers :
1 - Configure the ALM settings in SonarQube admin settings (ALM URL, token/credentials)
2 - Configure the ALM settings at the project level (ALM selection, ALM project key/id) 3 - Configure the CI to have the scanner running with the required parameters
This MMF is focusing on the first step only, because this is the most common one across all the ALMs and where we see problems raised by users. This should help to reduce the "surface area" of possible issues.
This problem of settings validation is true for the 4 following ALM (Application Lifecycle Management)
- Github Enterprise (GHE) & Cloud
- Bitbucket Server
- Azure Devops
So the admin settings validation should cover the 4 of them.
ALM configuration: Settings filled in the SonarQube admin panel.
ALM integration: Configuration and technical environment (proxy, etc).
In order of priority, use cases are :
- As an instance admin: Configure a new ALM
- As a SonarQube administrator, I want to enable the integration between SonarQube and an ALM. I want to know if my configuration is valid (see Validation section).
- I still want to be able to save an invalid configuration to be able to come back to it later (while I fix problems that can be outside of SQ).
- As an instance admin: Troubleshoot an existing ALM integration
- Is my integration valid (see Validation section) or not? Feedback should be provided to the user if SonarQube is able to reach the ALM.
- Trigger validation of a specific ALM integration
- As an instance admin: I want to know which features are enabled for an ALM and if not, get feedback about why they are not available.
It should be kept in mind (although we do not necessarily want to cover it right away) those 2 use cases for configuration of ALM at project level :
- As a project creator: I should not be able to onboard a project from an invalid ALM configuration: invalid ALM configuration should not be listed.
- As a project administrator: I should not be able to configure PR decoration for a project by using an invalid ALM configuration. Invalid configuration should not be selectable.
We want to validate :
- Authentication :
- Is the API endpoint correct? (ie: validating the url set is correct, reachable)
- Are the tokens recognized and valid? (ie: token provided by the user allows proper authentication)
- Has the token set by the user the proper level of permission required for SonarQube needs (PR decoration and Onboarding) ?
- Read access to a list of repositories
- Write access on PR
We want to be able to save an invalid configuration, but, as long as a configuration is not marked as valid by our checks, it should not be usable to import projects (set aside PR decoration for now).
Out of scope
- We don’t want to cover API version verification in the scope of this MMF.
- validating configuration of SSO authentication with ALM
To warn the SonarQube admin that something is wrong is an ALM integration, we want to show, in the “ALM Integration configurations” admin setting page, the current status of the integration for every ALM configured. This information can be updated at will, by refreshing the page. Every time a configuration is created/updated, status is refreshed as well.
Add a new api endpoint common for all 4 ALMs POST api/alm_settings/validate?key=xxx
Details of the endpoint implementation are in linked tickets bellow.
For DE we will have the same interface. The button "Add integration" will be inactive once one GitHub/Enteprise integration has been configured. A tooltip with following message will appear on hover "Upgrade to [Enterprise Edition](same link used in Marketplace) to integrate with multiple [ALM] instances." Replace ALM by GitHub, Gitlab... depending on the context.