Uploaded image for project: 'SonarQube'
  1. SonarQube
  2. SONAR-9909

Slow indexing of portfolios at server startup

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.6-M3, 6.6
    • Component/s: None
    • Labels:

      Description

      While indexing a 200M issues PostgreSQL DB in Elasticsearch, RDS CPU is very high.

      A look on PGSQL side show a very long running request (35mn) :

      12550 | -00:35:00.874108 | sonarsource | SELECT p.uuid FROM projects technical_projects                                                                                                                  
             |                  |             |     INNER JOIN projects p on p.uuid=technical_projects.copy_component_uuid AND p.enabled=true                                                                   
             |                  |             |     where                                                                                                                                                       
             |                  |             |       technical_projects.enabled=true                                                                                                                           
             |                  |             |       AND technical_projects.project_uuid = $1                                                                                                                  
             |                  |             |       AND technical_projects.module_uuid_path LIKE $2 
      
       12550 | -00:41:56.946158 | sonarsource | SELECT p.uuid FROM projects technical_projects                                                                                                                  
             |                  |             |     INNER JOIN projects p on p.uuid=technical_projects.copy_component_uuid AND p.enabled=true                                                                   
             |                  |             |     where                                                                                                                                                       
             |                  |             |       technical_projects.enabled=true                                                                                                                           
             |                  |             |       AND technical_projects.project_uuid = $1                                                                                                                  
             |                  |             |       AND technical_projects.module_uuid_path LIKE $2                                                                                                           
      

      While indexing ... views

      2017.10.03 14:07:08 INFO  web[][o.s.s.e.IndexerStartupTask] Indexing of type [views/view] ...
      2017.10.03 14:08:08 INFO  web[][o.s.s.es.BulkIndexer] 2 requests processed (0 items/sec)
      2017.10.03 14:09:08 INFO  web[][o.s.s.es.BulkIndexer] 2 requests processed (0 items/sec)
      2017.10.03 14:10:08 INFO  web[][o.s.s.es.BulkIndexer] 2 requests processed (0 items/sec)
      ....
      2017.10.03 18:10:08 INFO  web[][o.s.s.es.BulkIndexer] 21 requests processed (0 items/sec)
      2017.10.03 18:11:08 INFO  web[][o.s.s.es.BulkIndexer] 21 requests processed (0 items/sec)
      2017.10.03 18:12:08 INFO  web[][o.s.s.es.BulkIndexer] 21 requests processed (0 items/sec)
      2017.10.03 18:13:08 INFO  web[][o.s.s.es.BulkIndexer] 21 requests processed (0 items/sec)
      

        Attachments

          Activity

            People

            • Assignee:
              henri.gomez Henri Gomez (Inactive)
              Reporter:
              simon.brandhof Simon Brandhof
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Due:
                Created:
                Updated:
                Resolved: