Uploaded image for project: 'SonarQube'
  1. SonarQube
  2. SONAR-9216

Memory hotspot when indexing components at server startup

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 6.4
    • Fix Version/s: 6.5-M3, 6.5
    • Component/s: ElasticSearch
    • Labels:

      Description

      Indexing of components at startup loads all the components in memory. That can be huge on big instances, for instance sonarqube.com can't be loaded with default memory settings.

      2017.05.11 14:46:15 INFO  web[][o.s.s.q.RegisterQualityProfiles] Register quality profiles
      2017.05.11 14:46:15 INFO  web[][o.s.s.s.RegisterPermissionTemplates] Register permission templates
      2017.05.11 14:46:15 INFO  web[][o.s.s.s.RenameDeprecatedPropertyKeys] Rename deprecated property keys
      2017.05.11 14:46:15 INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.WebServiceFilter@5bb28ec9 [pattern=UrlPattern{inclusions=[/api/measures/component/*, ...], exclusions=[/api/properties*, ...]}]
      2017.05.11 14:46:15 INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.DeprecatedPropertiesWsFilter@630dce31 [pattern=UrlPattern{inclusions=[/api/properties/*], exclusions=[]}]
      2017.05.11 14:46:15 INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.WebServiceReroutingFilter@37cd191e [pattern=UrlPattern{inclusions=[/api/components/bulk_update_key, ...], exclusions=[]}]
      2017.05.11 14:46:15 INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.InitFilter@328abceb [pattern=UrlPattern{inclusions=[/sessions/init/*], exclusions=[]}]
      2017.05.11 14:46:15 INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.OAuth2CallbackFilter@7290f15d [pattern=UrlPattern{inclusions=[/oauth2/callback/*], exclusions=[]}]
      2017.05.11 14:46:15 INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.ws.LoginAction@4ed1d61e [pattern=UrlPattern{inclusions=[/api/authentication/login], exclusions=[]}]
      2017.05.11 14:46:15 INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.ws.LogoutAction@46c9d0f7 [pattern=UrlPattern{inclusions=[/api/authentication/logout], exclusions=[]}]
      2017.05.11 14:46:15 INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.ws.ValidateAction@75918990 [pattern=UrlPattern{inclusions=[/api/authentication/validate], exclusions=[]}]
      2017.05.11 14:46:15 INFO  web[][o.s.s.e.IndexerStartupTask] Indexing of types [rules/rule],[rules/ruleExtension] ...
      2017.05.11 14:46:26 INFO  web[][o.s.s.e.IndexerStartupTask] Indexing of types [rules/rule],[rules/ruleExtension] done | time=10625ms
      2017.05.11 14:46:26 INFO  web[][o.s.s.e.IndexerStartupTask] Indexing of type [users/user] ...
      2017.05.11 14:46:37 INFO  web[][o.s.s.e.IndexerStartupTask] Indexing of type [users/user] done | time=11116ms
      2017.05.11 14:46:37 INFO  web[][o.s.s.e.IndexerStartupTask] Indexing of type [components/component] ...
      2017.05.11 14:47:37 INFO  web[][o.s.s.es.BulkIndexer] 0 requests processed (0 items/sec)
      2017.05.11 14:48:48 INFO  web[][o.s.s.es.BulkIndexer] 0 requests processed (0 items/sec)
      2017.05.11 14:48:49 ERROR web[][o.s.s.p.Platform] Background initialization failed. Stopping SonarQube
      java.lang.OutOfMemoryError: GC overhead limit exceeded
      	at java.util.zip.ZipCoder.getBytes(ZipCoder.java:80)
      	at java.util.zip.ZipFile.getEntry(ZipFile.java:310)
      	at java.util.jar.JarFile.getEntry(JarFile.java:240)
      	at java.util.jar.JarFile.getJarEntry(JarFile.java:223)
      	at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:1042)
      	at sun.misc.URLClassPath.getResource(URLClassPath.java:239)
      	at java.net.URLClassLoader$1.run(URLClassLoader.java:365)
      	at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1989)
      	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:217)
      	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:421)
      	at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:166)
      	at org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:159)
      	at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
      	at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
      	at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(PreparedStatementHandler.java:59)
      	at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(RoutingStatementHandler.java:73)
      	at org.apache.ibatis.executor.ReuseExecutor.doQuery(ReuseExecutor.java:57)
      	at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:267)
      	at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:137)
      	at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:96)
      	at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:77)
      	at org.apache.ibatis.session.defaults.DefaultSqlSession.select(DefaultSqlSession.java:128)
      	at org.apache.ibatis.session.defaults.DefaultSqlSession.select(DefaultSqlSession.java:118)
      	at org.apache.ibatis.binding.MapperMethod.executeWithResultHandler(MapperMethod.java:108)
      	at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:60)
      	at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:52)
      2017.05.11 14:48:49 INFO  web[][o.s.p.StopWatcher] Stopping process
      

      Problem comes from usage of MyBatis ResultHandler ComponentDao#selectForIndexing(DbSession session, @Nullable String projectUuid, ResultHandler handler). Items seem to be kept in memory. See Dialect#getScrollDefaultFetchSize() for possible solution.

        Attachments

          Activity

            People

            • Assignee:
              simon.brandhof Simon Brandhof
              Reporter:
              simon.brandhof Simon Brandhof
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: