-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 6.4
-
Component/s: ElasticSearch
-
Labels:
Indexing of components at startup loads all the components in memory. That can be huge on big instances, for instance sonarqube.com can't be loaded with default memory settings.
2017.05.11 14:46:15 INFO web[][o.s.s.q.RegisterQualityProfiles] Register quality profiles
2017.05.11 14:46:15 INFO web[][o.s.s.s.RegisterPermissionTemplates] Register permission templates
2017.05.11 14:46:15 INFO web[][o.s.s.s.RenameDeprecatedPropertyKeys] Rename deprecated property keys
2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.WebServiceFilter@5bb28ec9 [pattern=UrlPattern{inclusions=[/api/measures/component/*, ...], exclusions=[/api/properties*, ...]}]
2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.DeprecatedPropertiesWsFilter@630dce31 [pattern=UrlPattern{inclusions=[/api/properties/*], exclusions=[]}]
2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.WebServiceReroutingFilter@37cd191e [pattern=UrlPattern{inclusions=[/api/components/bulk_update_key, ...], exclusions=[]}]
2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.InitFilter@328abceb [pattern=UrlPattern{inclusions=[/sessions/init/*], exclusions=[]}]
2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.OAuth2CallbackFilter@7290f15d [pattern=UrlPattern{inclusions=[/oauth2/callback/*], exclusions=[]}]
2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.ws.LoginAction@4ed1d61e [pattern=UrlPattern{inclusions=[/api/authentication/login], exclusions=[]}]
2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.ws.LogoutAction@46c9d0f7 [pattern=UrlPattern{inclusions=[/api/authentication/logout], exclusions=[]}]
2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.ws.ValidateAction@75918990 [pattern=UrlPattern{inclusions=[/api/authentication/validate], exclusions=[]}]
2017.05.11 14:46:15 INFO web[][o.s.s.e.IndexerStartupTask] Indexing of types [rules/rule],[rules/ruleExtension] ...
2017.05.11 14:46:26 INFO web[][o.s.s.e.IndexerStartupTask] Indexing of types [rules/rule],[rules/ruleExtension] done | time=10625ms
2017.05.11 14:46:26 INFO web[][o.s.s.e.IndexerStartupTask] Indexing of type [users/user] ...
2017.05.11 14:46:37 INFO web[][o.s.s.e.IndexerStartupTask] Indexing of type [users/user] done | time=11116ms
2017.05.11 14:46:37 INFO web[][o.s.s.e.IndexerStartupTask] Indexing of type [components/component] ...
2017.05.11 14:47:37 INFO web[][o.s.s.es.BulkIndexer] 0 requests processed (0 items/sec)
2017.05.11 14:48:48 INFO web[][o.s.s.es.BulkIndexer] 0 requests processed (0 items/sec)
2017.05.11 14:48:49 ERROR web[][o.s.s.p.Platform] Background initialization failed. Stopping SonarQube
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.zip.ZipCoder.getBytes(ZipCoder.java:80)
at java.util.zip.ZipFile.getEntry(ZipFile.java:310)
at java.util.jar.JarFile.getEntry(JarFile.java:240)
at java.util.jar.JarFile.getJarEntry(JarFile.java:223)
at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:1042)
at sun.misc.URLClassPath.getResource(URLClassPath.java:239)
at java.net.URLClassLoader$1.run(URLClassLoader.java:365)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1989)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:217)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:421)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:166)
at org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:159)
at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(PreparedStatementHandler.java:59)
at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(RoutingStatementHandler.java:73)
at org.apache.ibatis.executor.ReuseExecutor.doQuery(ReuseExecutor.java:57)
at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:267)
at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:137)
at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:96)
at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:77)
at org.apache.ibatis.session.defaults.DefaultSqlSession.select(DefaultSqlSession.java:128)
at org.apache.ibatis.session.defaults.DefaultSqlSession.select(DefaultSqlSession.java:118)
at org.apache.ibatis.binding.MapperMethod.executeWithResultHandler(MapperMethod.java:108)
at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:60)
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:52)
2017.05.11 14:48:49 INFO web[][o.s.p.StopWatcher] Stopping process
Problem comes from usage of MyBatis ResultHandler ComponentDao#selectForIndexing(DbSession session, @Nullable String projectUuid, ResultHandler handler). Items seem to be kept in memory. See Dialect#getScrollDefaultFetchSize() for possible solution.