Uploaded image for project: 'SonarQube'
  1. SonarQube
  2. SONAR-15259

Refactor storage of Portfolios from XML to DB



    • Type: Task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 9.1
    • Component/s: None
    • Labels:
    • Edition:
    • Production Notes:


      Current storage of Portfolios

      Portfolios are all stored in a single internal property, encoded in XML. In addition to that, as any other component in SonarQube, it's also stored in the Components table.

      Portfolios have been moved out of the XML storage and stored in the Projects and Project_branches tables.

      The goal of this ticket is to store Portfolios in the database using properly normalized tables.

      It should provide efficient data storage and allow us to perform all needed operations on portfolios in an efficient way. 

      View Definition

      A definition of a portfolio is composed of data that can be split into 3 categories:

      Project selection:

      • isDefault
      • List of projects
      • Regexp
      • Tags association


      • Key (already in ProjectDto)
      • Name (already in ProjectDto)
      • Description (already in ProjectDto)
      • Qualifier (already in ProjectDto)


      • Parent
      • Root
      • List of view references ← can be applications


      • Tag key  ← Not used anymore
      • Tag value  ← Not used anymore
      • Language←Not used anymore
      • List application branches  ← Not used anymore. We still use ApplicationBranchDefinition and ApplicationProjectDefinition just to export with DefinitionAction

      Main classes

      • DefinitionsXmlLoader: loads xml from internal properties
      • DefinitionsLoaderImpl: creates Definitions parsing xml that it gets with DefinitionsXmlLoader
      • ViewDefinition: single view definition
      • Definitions: keep roots of the tree and data about the hierarchy
      • ViewsXmlUpdater: uses Definitions to do changes and saves to internal properties


      Operations supported

      This list is probably not complete.

      • Define Portfolio structure
      • Add/Delete portfolio in the hierarchy
      • Move in the hierarchy
      • Trigger refresh of part of the tree when needed
      • List all portfolios
      • Efficiently find the root portfolio
      • Define references
      • Add/Delete references to portfolios or apps
      • Detect Cycles, taking into account references. IIRC, easiest way is to look at the root of references
      • Find all portfolios referencing a certain app/portfolio
      • Project Selection
      • Get/Set new project selection
      • Efficiently resolve selection to a list of projects
      • Import/Export
      • Export/Import XML definitions of portfolios and applications

      DB Schema

      Store portfolios in the table 'Projects'. Additionally, create the following tables:

      * Portfolio_references
      - uuid
      - portfolio_uuid
      - reference_uuid
      * Portfolio_projects
      - uuid
      - portfolio_uuid
      - project_uuid
      * Portfolios
      - uuid
      - key
      - name
      - description
      - root_uuid
      - parent_uuid
      - portfolio_uuid
      - selection_mode
      - selection_expression


      TBD: define indexes and PKs


      1. Create Portfolios table
      2. Create table to store Portfolio hierarchy
      3. Create table to store Portfolio references to other portfolios and applications
      4. Refactor all the classes listed above to become Helper classes that support common operations requiring access to multiple DAOs of the new tables
      5. Verify that Purge cleans new tables
      6. Write migration to move data from the internal property to the DB
      7. Rewrite Define/Definition WS to bridge between XML and DB (already done for apps)




            duarte.meneses Duarte Meneses
            duarte.meneses Duarte Meneses
            0 Vote for this issue
            2 Start watching this issue