Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 6.4
-
Component/s: ElasticSearch
-
Labels:
Description
Indexing of components at startup loads all the components in memory. That can be huge on big instances, for instance sonarqube.com can't be loaded with default memory settings.
2017.05.11 14:46:15 INFO web[][o.s.s.q.RegisterQualityProfiles] Register quality profiles 2017.05.11 14:46:15 INFO web[][o.s.s.s.RegisterPermissionTemplates] Register permission templates 2017.05.11 14:46:15 INFO web[][o.s.s.s.RenameDeprecatedPropertyKeys] Rename deprecated property keys 2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.WebServiceFilter@5bb28ec9 [pattern=UrlPattern{inclusions=[/api/measures/component/*, ...], exclusions=[/api/properties*, ...]}] 2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.DeprecatedPropertiesWsFilter@630dce31 [pattern=UrlPattern{inclusions=[/api/properties/*], exclusions=[]}] 2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.WebServiceReroutingFilter@37cd191e [pattern=UrlPattern{inclusions=[/api/components/bulk_update_key, ...], exclusions=[]}] 2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.InitFilter@328abceb [pattern=UrlPattern{inclusions=[/sessions/init/*], exclusions=[]}] 2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.OAuth2CallbackFilter@7290f15d [pattern=UrlPattern{inclusions=[/oauth2/callback/*], exclusions=[]}] 2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.ws.LoginAction@4ed1d61e [pattern=UrlPattern{inclusions=[/api/authentication/login], exclusions=[]}] 2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.ws.LogoutAction@46c9d0f7 [pattern=UrlPattern{inclusions=[/api/authentication/logout], exclusions=[]}] 2017.05.11 14:46:15 INFO web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.ws.ValidateAction@75918990 [pattern=UrlPattern{inclusions=[/api/authentication/validate], exclusions=[]}] 2017.05.11 14:46:15 INFO web[][o.s.s.e.IndexerStartupTask] Indexing of types [rules/rule],[rules/ruleExtension] ... 2017.05.11 14:46:26 INFO web[][o.s.s.e.IndexerStartupTask] Indexing of types [rules/rule],[rules/ruleExtension] done | time=10625ms 2017.05.11 14:46:26 INFO web[][o.s.s.e.IndexerStartupTask] Indexing of type [users/user] ... 2017.05.11 14:46:37 INFO web[][o.s.s.e.IndexerStartupTask] Indexing of type [users/user] done | time=11116ms 2017.05.11 14:46:37 INFO web[][o.s.s.e.IndexerStartupTask] Indexing of type [components/component] ... 2017.05.11 14:47:37 INFO web[][o.s.s.es.BulkIndexer] 0 requests processed (0 items/sec) 2017.05.11 14:48:48 INFO web[][o.s.s.es.BulkIndexer] 0 requests processed (0 items/sec) 2017.05.11 14:48:49 ERROR web[][o.s.s.p.Platform] Background initialization failed. Stopping SonarQube java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.zip.ZipCoder.getBytes(ZipCoder.java:80) at java.util.zip.ZipFile.getEntry(ZipFile.java:310) at java.util.jar.JarFile.getEntry(JarFile.java:240) at java.util.jar.JarFile.getJarEntry(JarFile.java:223) at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:1042) at sun.misc.URLClassPath.getResource(URLClassPath.java:239) at java.net.URLClassLoader$1.run(URLClassLoader.java:365) at java.net.URLClassLoader$1.run(URLClassLoader.java:362) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:361) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1989) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:217) at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:421) at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:166) at org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:159) at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172) at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172) at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(PreparedStatementHandler.java:59) at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(RoutingStatementHandler.java:73) at org.apache.ibatis.executor.ReuseExecutor.doQuery(ReuseExecutor.java:57) at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:267) at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:137) at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:96) at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:77) at org.apache.ibatis.session.defaults.DefaultSqlSession.select(DefaultSqlSession.java:128) at org.apache.ibatis.session.defaults.DefaultSqlSession.select(DefaultSqlSession.java:118) at org.apache.ibatis.binding.MapperMethod.executeWithResultHandler(MapperMethod.java:108) at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:60) at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:52) 2017.05.11 14:48:49 INFO web[][o.s.p.StopWatcher] Stopping process
Problem comes from usage of MyBatis ResultHandler ComponentDao#selectForIndexing(DbSession session, @Nullable String projectUuid, ResultHandler handler). Items seem to be kept in memory. See Dialect#getScrollDefaultFetchSize() for possible solution.