Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 6.4
-
Component/s: ElasticSearch
-
Labels:
Description
Indexing of components at startup loads all the components in memory. That can be huge on big instances, for instance sonarqube.com can't be loaded with default memory settings.
2017.05.11 14:46:15 INFO web[][o.s.s.q.RegisterQualityProfiles] Register quality profiles
2017.05.11 14:46:15 INFO web[][o.s.s.s.RegisterPermissionTemplates] Register permission templates
2017.05.11 14:46:15 INFO web[][o.s.s.s.RenameDeprecatedPropertyKeys] Rename deprecated property keys
2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.WebServiceFilter@5bb28ec9 [pattern=UrlPattern{inclusions=[/api/measures/component/*, ...], exclusions=[/api/properties*, ...]}]
2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.DeprecatedPropertiesWsFilter@630dce31 [pattern=UrlPattern{inclusions=[/api/properties/*], exclusions=[]}]
2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.WebServiceReroutingFilter@37cd191e [pattern=UrlPattern{inclusions=[/api/components/bulk_update_key, ...], exclusions=[]}]
2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.InitFilter@328abceb [pattern=UrlPattern{inclusions=[/sessions/init/*], exclusions=[]}]
2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.OAuth2CallbackFilter@7290f15d [pattern=UrlPattern{inclusions=[/oauth2/callback/*], exclusions=[]}]
2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.ws.LoginAction@4ed1d61e [pattern=UrlPattern{inclusions=[/api/authentication/login], exclusions=[]}]
2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.ws.LogoutAction@46c9d0f7 [pattern=UrlPattern{inclusions=[/api/authentication/logout], exclusions=[]}]
2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.ws.ValidateAction@75918990 [pattern=UrlPattern{inclusions=[/api/authentication/validate], exclusions=[]}]
2017.05.11 14:46:15 INFO web[][o.s.s.e.IndexerStartupTask] Indexing of types [rules/rule],[rules/ruleExtension] ...
2017.05.11 14:46:26 INFO web[][o.s.s.e.IndexerStartupTask] Indexing of types [rules/rule],[rules/ruleExtension] done | time=10625ms
2017.05.11 14:46:26 INFO web[][o.s.s.e.IndexerStartupTask] Indexing of type [users/user] ...
2017.05.11 14:46:37 INFO web[][o.s.s.e.IndexerStartupTask] Indexing of type [users/user] done | time=11116ms
2017.05.11 14:46:37 INFO web[][o.s.s.e.IndexerStartupTask] Indexing of type [components/component] ...
2017.05.11 14:47:37 INFO web[][o.s.s.es.BulkIndexer] 0 requests processed (0 items/sec)
2017.05.11 14:48:48 INFO web[][o.s.s.es.BulkIndexer] 0 requests processed (0 items/sec)
2017.05.11 14:48:49 ERROR web[][o.s.s.p.Platform] Background initialization failed. Stopping SonarQube
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.zip.ZipCoder.getBytes(ZipCoder.java:80)
at java.util.zip.ZipFile.getEntry(ZipFile.java:310)
at java.util.jar.JarFile.getEntry(JarFile.java:240)
at java.util.jar.JarFile.getJarEntry(JarFile.java:223)
at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:1042)
at sun.misc.URLClassPath.getResource(URLClassPath.java:239)
at java.net.URLClassLoader$1.run(URLClassLoader.java:365)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1989)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:217)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:421)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:166)
at org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:159)
at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(PreparedStatementHandler.java:59)
at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(RoutingStatementHandler.java:73)
at org.apache.ibatis.executor.ReuseExecutor.doQuery(ReuseExecutor.java:57)
at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:267)
at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:137)
at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:96)
at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:77)
at org.apache.ibatis.session.defaults.DefaultSqlSession.select(DefaultSqlSession.java:128)
at org.apache.ibatis.session.defaults.DefaultSqlSession.select(DefaultSqlSession.java:118)
at org.apache.ibatis.binding.MapperMethod.executeWithResultHandler(MapperMethod.java:108)
at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:60)
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:52)
2017.05.11 14:48:49 INFO web[][o.s.p.StopWatcher] Stopping process
Problem comes from usage of MyBatis ResultHandler ComponentDao#selectForIndexing(DbSession session, @Nullable String projectUuid, ResultHandler handler). Items seem to be kept in memory. See Dialect#getScrollDefaultFetchSize() for possible solution.