Better scalability of loading of project measures

Loading measures from database can be quite slow when many projects have been analyzed or when projects have a long history.

Current state

Future structure

What needs to be done?

• Create table live_measures
• Migrate all data from project_measures, that relate to a latest analysis of a project
• Let the compute engine store only project level data into project_measures, and all levels into live_measures
• Make web services read from live_measures instead of project_measures. Some examples:
  • All issue update web services (when generating the response)
  • api/measures/component
  • api/measures/component_tree
  • api/measures/search
  • api/qualitygate/project_status
  • api/branches/list
  • api/duplications/show
  • api/ui/component
• Update purge mechanisms:
  • Deletion of non-project level data, that is not related to a last analysis
  • Deletion of metrics (if such mechanism exists)
  • Deletion of projects
  • Deletion of organization
• Update the governance plugin:
  • import of project dump
  • computation of applications and portfolios
• Update the branch plugin

After everything is updated, a last step will be:

• Update the new database migration step, to also delete all non-project related data from project_measures

Elasticsearch indexing has not to be changed right now, because only project level data is indexed (and that data will remain in both tables)