As a manager following the quality of a set of projects, I'm interested to know if these projects are on track with the quality policy of the company and where my risks are.
This is what is currently provided by the portfolio feature (available from EE) showing the health (the ratings) and the risk (number of projects having the worst rating).
The Security Hotspot issue type was introduced and developers are invited to review them on regular basis to assess if there are risks to use this or that security-sensitive API.
At portfolio level, there is no indicator of portfolio health with regard to this review. Instead, we need to raise awareness about the state of the review.
If all the Security Vulnerability issues have been fixed on two portfolios and that portfolio 1 has 500 Security Hotspots open and portfolio 2 is having 2 Security Hotspots, which one is more at risk? This is what we need to solve in this MMF.
Let's introduce a new metric called Security Review Rating about the ratio of To Review|In Review Security Hotspots per 1K LOC. We'll use the standard rating scale:
- A: <=3 To Review|In Review Hotspots/1k LOC
- B: >3-10
- C: >10-15
- D: >15-25
- E: >25
This metric is not calculated for files, but only at project level and above (i.e. +Portfolios & Applications).
We must expose the Security Review Rating to executives in:
- portfolio homepage - add risk with the same treatment as other ratings.
- portfolio PDF
The rating will also be exposed at portfolio level in:
- Security report - in the header, not per category. This will be net hotspots/1k LoC, so summing numbers visible on the report may not yield the same underlying ratio as reflected here.
As with other ratings, a Portfolio's Security Review Rating is the average of the underlying projects' ratings.
At the project level, the measure will be exposed in the redesigned Security Report.
Wherever we expose this rating, we must also make available an explanation of hotspots and of what this Security Review Rating means.
Immediately after upgrade there will be a period where the new metric has not yet been calculated. On the project homepage and in the security reports, Measures and Activity, the best course is to simply omit the rating until it is available. On the portfolio homepage, where the omission will be much more obvious, we should sub-in a dash ('-'). The trend and risk lines will simply be omitted in this case.
*Tooltip content is under creation.
Micro-interactions, tooltips, links are visible in the following prototype: https://invis.io/QAS86M4J42P#/365653902_Portfolio_Overview
Please activate comment mode (bottom right of the screen) to see the links and tooltip contents.
- Security Hotspots Review rating is displayed on the Portfolio Overview page at the same level of other overall health factors.
- Portfolio Breakdown has a new column "Security Hotspots Review" with the new rating at project/sub-portfolio level.
- We introduce a new icon for Measures. **Icon attached to the MMF**
- Title of cards (e.g Portfolio health factors) font: 16px Light
- The portfolio pdf is now accessible via a dropdown (see prototype to see micro-interactions and notifications content).
Tool tips for each card:
Releasability: Ratio of projects in the Portfolio that have passed the Quality Gate.
Reliability: Average Reliability rating for all projects in the portfolio.
Security Vulnerabilities: Average security rating for all projects in the portfolio.
Security Hotspot Review: Ratio of To Review or In Review Security Hotspots per 1k lines of code.
Maintainability: Average maintainability rating for all projects in the portfolio.
- Security reports are now grouped into one single page. They are displayed in tabs.
- The column "won't fix" is removed from the reports.
- The security metrics are visible on the top right of the page.
1 new tooltip:
Categories tooltip Content: "_Vulnerabilities and Hotspots may map to multiple categories. For this reason the total number of Vulnerabilities and Security Hotspots in this report might be superior to project metrics."