Uploaded image for project: 'SonarQube'
  1. SonarQube
  2. SONAR-6995

Server fails to start because of error on "initPendingCount"

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 5.2
    • Fix Version/s: 5.3
    • Component/s: None
    • Labels:
      None

      Description

      Got this error once with 5.2 GA on a mysql database which was already migrated to 5.2:

      2015.11.03 14:29:43 TRACE web[o.s.c.p.ComponentContainer] org.sonar.server.batch.BatchIndex started | time=124ms
      2015.11.03 14:29:43 TRACE web[o.s.c.p.ComponentContainer] org.sonar.server.issue.index.IssueAuthorizationIndexer started | time=0ms
      2015.11.03 14:29:43 TRACE web[o.s.c.p.ComponentContainer] org.sonar.server.issue.index.IssueIndexer started | time=0ms
      2015.11.03 14:29:43 TRACE web[o.s.c.p.ComponentContainer] org.sonar.server.test.index.TestIndexer started | time=0ms
      2015.11.03 14:29:43 TRACE web[o.s.c.p.ComponentContainer] org.sonar.core.issue.workflow.IssueWorkflow started | time=5ms
      2015.11.03 14:29:43 TRACE web[o.s.c.p.ComponentContainer] org.sonar.server.ws.WebServiceEngine started | time=0ms
      2015.11.03 14:29:43 TRACE web[o.s.c.p.ComponentContainer] org.sonar.server.view.index.ViewIndexer started | time=0ms
      2015.11.03 14:29:43 INFO  web[o.s.s.n.NotificationService] Notification service started (delay 60 sec.)
      2015.11.03 14:29:43 TRACE web[o.s.c.p.ComponentContainer] org.sonar.server.notification.NotificationService started | time=1ms
      2015.11.03 14:29:43 TRACE web[o.s.c.p.ComponentContainer] org.sonar.server.view.bridge.ViewsStopper started | time=0ms
      2015.11.03 14:29:43 TRACE web[o.s.c.p.ComponentContainer] org.sonar.server.computation.taskprocessor.CeProcessingSchedulerExecutorServiceImpl started | time=0ms
      2015.11.03 14:29:43 TRACE web[sql] time=0ms | sql=select id, data    from notifications    order by id asc    limit ?
      2015.11.03 14:29:43 TRACE web[sql] time=16ms | sql=select count(id) from ce_queue
      2015.11.03 14:29:43 INFO  web[o.s.s.n.NotificationService] Notification service stopped
      2015.11.03 14:29:43 ERROR web[o.a.c.c.C.[.[.[/]] Exception sending context initialized event to listener instance of class org.sonar.server.platform.PlatformServletContextListener
      java.lang.IllegalStateException: Method initPendingCount must be used before any other method and can not be called twice
              at org.sonar.server.computation.monitoring.CEQueueStatusImpl.initPendingCount(CEQueueStatusImpl.java:54) ~[sonar-server-5.2-M66_2015-10-30.jar:na]
              at org.sonar.server.computation.queue.CeQueueInitializer.initJmxCounters(CeQueueInitializer.java:68) ~[sonar-server-5.2-M66_2015-10-30.jar:na]
              at org.sonar.server.computation.queue.CeQueueInitializer.start(CeQueueInitializer.java:53) ~[sonar-server-5.2-M66_2015-10-30.jar:na]
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_65]
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_65]
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_65]
              at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_65]
              at org.picocontainer.lifecycle.ReflectionLifecycleStrategy.invokeMethod(ReflectionLifecycleStrategy.java:110) ~[picocontainer-2.15.jar:na]
              at org.picocontainer.lifecycle.ReflectionLifecycleStrategy.start(ReflectionLifecycleStrategy.java:89) ~[picocontainer-2.15.jar:na]
              at org.sonar.core.platform.ComponentContainer$1.start(ComponentContainer.java:259) ~[sonar-core-5.2-M66_2015-10-30.jar:na]
              at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.start(AbstractInjectionFactory.java:84) ~[picocontainer-2.15.jar:na]
              at org.picocontainer.behaviors.AbstractBehavior.start(AbstractBehavior.java:169) ~[picocontainer-2.15.jar:na]
              at org.picocontainer.behaviors.Stored$RealComponentLifecycle.start(Stored.java:132) ~[picocontainer-2.15.jar:na]
              at org.picocontainer.behaviors.Stored.start(Stored.java:110) ~[picocontainer-2.15.jar:na]
              at org.picocontainer.DefaultPicoContainer.potentiallyStartAdapter(DefaultPicoContainer.java:1016) ~[picocontainer-2.15.jar:na]
              at org.picocontainer.DefaultPicoContainer.startAdapters(DefaultPicoContainer.java:1009) ~[picocontainer-2.15.jar:na]
              at org.picocontainer.DefaultPicoContainer.start(DefaultPicoContainer.java:767) ~[picocontainer-2.15.jar:na]
              at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:99) ~[sonar-core-5.2-M66_2015-10-30.jar:na]
              at org.sonar.server.platform.platformlevel.PlatformLevel.start(PlatformLevel.java:84) ~[sonar-server-5.2-M66_2015-10-30.jar:na]
              at org.sonar.server.platform.platformlevel.PlatformLevel4.start(PlatformLevel4.java:729) ~[sonar-server-5.2-M66_2015-10-30.jar:na]
              at org.sonar.server.platform.Platform.start(Platform.java:214) ~[sonar-server-5.2-M66_2015-10-30.jar:na]
              at org.sonar.server.platform.Platform.startLevel34Containers(Platform.java:188) ~[sonar-server-5.2-M66_2015-10-30.jar:na]
              at org.sonar.server.platform.Platform.doStart(Platform.java:113) ~[sonar-server-5.2-M66_2015-10-30.jar:na]
              at org.sonar.server.platform.Platform.doStart(Platform.java:99) ~[sonar-server-5.2-M66_2015-10-30.jar:na]
              at org.sonar.server.platform.PlatformServletContextListener.contextInitialized(PlatformServletContextListener.java:43) ~[sonar-server-5.2-M66_2015-10-30.jar:na]
              at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4720) [tomcat-embed-core-8.0.18.jar:8.0.18]
              at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5154) [tomcat-embed-core-8.0.18.jar:8.0.18]
              at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) [tomcat-embed-core-8.0.18.jar:8.0.18]
              at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1409) [tomcat-embed-core-8.0.18.jar:8.0.18]
              at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1399) [tomcat-embed-core-8.0.18.jar:8.0.18]
              at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_65]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_65]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_65]
              at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
      2015.11.03 14:29:43 INFO  web[jruby.rack] jruby 1.7.9 (ruby-1.8.7p370) 2013-12-06 87b108a on Java HotSpot(TM) 64-Bit Server VM 1.8.0_65-b17 [linux-amd64]
      2015.11.03 14:29:43 INFO  web[jruby.rack] using a shared (threadsafe!) runtime
      2015.11.03 14:29:47 ERROR web[jruby.rack] initialization failed
      org.jruby.rack.RackInitializationException: java.lang.NullPointerException
              at org.jruby.rack.RackInitializationException.wrap(RackInitializationException.java:31) ~[jruby-rack-1.1.13.2.jar:na]
              at org.jruby.rack.RackApplicationFactoryDecorator.init(RackApplicationFactoryDecorator.java:98) ~[jruby-rack-1.1.13.2.jar:na]
              at org.jruby.rack.RackServletContextListener.contextInitialized(RackServletContextListener.java:50) ~[jruby-rack-1.1.13.2.jar:na]
              at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4720) [tomcat-embed-core-8.0.18.jar:8.0.18]
              at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5154) [tomcat-embed-core-8.0.18.jar:8.0.18]
              at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) [tomcat-embed-core-8.0.18.jar:8.0.18]
              at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1409) [tomcat-embed-core-8.0.18.jar:8.0.18]
              at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1399) [tomcat-embed-core-8.0.18.jar:8.0.18]
              at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_65]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_65]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_65]
              at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
      Caused by: java.lang.NullPointerException: null
              at org.sonar.server.platform.Platform.getContainer(Platform.java:282) ~[sonar-server-5.2-M66_2015-10-30.jar:na]
              at org.sonar.server.ui.JRubyFacade.getContainer(JRubyFacade.java:341) ~[sonar-server-5.2-M66_2015-10-30.jar:na]
              at org.sonar.server.ui.JRubyFacade.get(JRubyFacade.java:81) ~[sonar-server-5.2-M66_2015-10-30.jar:na]
              at org.sonar.server.ui.JRubyFacade.getDatabase(JRubyFacade.java:212) ~[sonar-server-5.2-M66_2015-10-30.jar:na]
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_65]
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_65]
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_65]
              at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_65]
              at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:440) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:304) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:52) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.RootNode.interpret(RootNode.java:129) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.evaluator.ASTInterpreter.INTERPRET_EVAL(ASTInterpreter.java:95) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.evaluator.ASTInterpreter.evalWithBinding(ASTInterpreter.java:184) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.RubyKernel.evalCommon(RubyKernel.java:1156) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.RubyKernel.eval(RubyKernel.java:1114) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.RubyKernel$INVOKER$s$0$3$eval.call(RubyKernel$INVOKER$s$0$3$eval.gen) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:180) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:296) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:72) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.FCallManyArgsNode.interpret(FCallManyArgsNode.java:60) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.IfNode.interpret(IfNode.java:118) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:139) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:182) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:139) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:182) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.AttrAssignOneArgNode.interpret(AttrAssignOneArgNode.java:33) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.IfNode.interpret(IfNode.java:116) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:139) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:182) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.VCallNode.interpret(VCallNode.java:88) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:161) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:190) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.RubyClass.finvoke(RubyClass.java:527) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.RubyBasicObject.send(RubyBasicObject.java:2590) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.RubyKernel.send(RubyKernel.java:2223) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.RubyKernel$INVOKER$s$send.call(RubyKernel$INVOKER$s$send.gen) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodOneOrNBlock.call(JavaMethod.java:350) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:326) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:170) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:161) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:190) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:316) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:145) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:154) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.CallNoArgBlockNode.interpret(CallNoArgBlockNode.java:64) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.RootNode.interpret(RootNode.java:129) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.evaluator.ASTInterpreter.INTERPRET_ROOT(ASTInterpreter.java:121) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.Ruby.runInterpreter(Ruby.java:838) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.Ruby.loadFile(Ruby.java:2727) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.load.ExternalScript.load(ExternalScript.java:66) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.load.LoadService.load(LoadService.java:359) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.RubyKernel.loadCommon(RubyKernel.java:1107) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.RubyKernel.load(RubyKernel.java:1094) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.RubyKernel$INVOKER$s$0$1$load.call(RubyKernel$INVOKER$s$0$1$load.gen) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:210) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:206) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:326) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:170) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:139) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:182) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:139) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:182) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.ast.RootNode.interpret(RootNode.java:129) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.evaluator.ASTInterpreter.INTERPRET_ROOT(ASTInterpreter.java:121) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.Ruby.evalScriptlet(Ruby.java:419) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.Ruby.evalScriptlet(Ruby.java:396) ~[jruby-complete-1.7.9.jar:na]
              at org.jruby.rack.rails.RailsRackApplicationFactory.createApplicationObject(RailsRackApplicationFactory.java:22) ~[jruby-rack-1.1.13.2.jar:na]
              at org.jruby.rack.DefaultRackApplicationFactory$1.create(DefaultRackApplicationFactory.java:98) ~[jruby-rack-1.1.13.2.jar:na]
              at org.jruby.rack.DefaultRackApplicationFactory$RackApplicationImpl.init(DefaultRackApplicationFactory.java:420) ~[jruby-rack-1.1.13.2.jar:na]
              at org.jruby.rack.DefaultRackApplicationFactory.getApplication(DefaultRackApplicationFactory.java:111) ~[jruby-rack-1.1.13.2.jar:na]
              at org.jruby.rack.SharedRackApplicationFactory.doInit(SharedRackApplicationFactory.java:31) ~[jruby-rack-1.1.13.2.jar:na]
              at org.jruby.rack.RackApplicationFactoryDecorator.init(RackApplicationFactoryDecorator.java:94) ~[jruby-rack-1.1.13.2.jar:na]
              ... 10 common frames omitted
      2015.11.03 14:29:47 ERROR web[o.a.c.c.StandardContext] Error listenerStart
      2015.11.03 14:29:47 ERROR web[o.a.c.c.StandardContext] Context [] startup failed due to previous errors
      2015.11.03 14:29:47 WARN  web[o.a.c.l.WebappClassLoaderBase] The web application [ROOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
       java.lang.Object.wait(Native Method)
       java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
       com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:43)
      2015.11.03 14:29:47 WARN  web[o.a.c.l.WebappClassLoaderBase] The web application [ROOT] appears to have started a thread named [JRubyJIT-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
       sun.misc.Unsafe.park(Native Method)
       java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
       java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
       java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
       java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
       java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
       java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
       java.lang.Thread.run(Thread.java:745)
      2015.11.03 14:29:47 WARN  web[o.a.c.l.WebappClassLoaderBase] The web application [ROOT] appears to have started a thread named [JRubyJIT-2] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
       sun.misc.Unsafe.park(Native Method)
       java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
       java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
       java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
       java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
       java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
       java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
       java.lang.Thread.run(Thread.java:745)
      2015.11.03 14:29:47 INFO  web[o.a.c.h.Http11NioProtocol] Starting ProtocolHandler ["http-nio-0.0.0.0-9000"]
      2015.11.03 14:29:47 INFO  web[o.s.s.a.TomcatAccessLog] Web server is started
      2015.11.03 14:29:47 INFO  web[o.s.s.a.EmbeddedTomcat] HTTP connector enabled on port 9000
      2015.11.03 14:29:47 WARN  web[o.s.p.ProcessEntryPoint] Fail to start web
      java.lang.IllegalStateException: Webapp did not start
              at org.sonar.server.app.EmbeddedTomcat.isReady(EmbeddedTomcat.java:83) ~[sonar-server-5.2-M66_2015-10-30.jar:na]
              at org.sonar.server.app.WebServer.isReady(WebServer.java:45) [sonar-server-5.2-M66_2015-10-30.jar:na]
              at org.sonar.process.ProcessEntryPoint.launch(ProcessEntryPoint.java:80) ~[sonar-process-5.2-M66_2015-10-30.jar:na]
              at org.sonar.server.app.WebServer.main(WebServer.java:66) [sonar-server-5.2-M66_2015-10-30.jar:na]
      2015.11.03 14:29:47 INFO  web[o.a.c.h.Http11NioProtocol] Pausing ProtocolHandler ["http-nio-0.0.0.0-9000"]
      2015.11.03 14:29:48 INFO  web[o.a.c.h.Http11NioProtocol] Stopping ProtocolHandler ["http-nio-0.0.0.0-9000"]
      2015.11.03 14:29:48 INFO  web[o.a.c.h.Http11NioProtocol] Destroying ProtocolHandler ["http-nio-0.0.0.0-9000"]
      2015.11.03 14:29:48 INFO  web[o.s.s.a.TomcatAccessLog] Web server is stopped
      2015.11.03 14:29:48 INFO  app[o.s.p.m.TerminatorThread] Process[search] is stopping
      2015.11.03 14:29:48 INFO   es[o.s.p.StopWatcher]  Stopping process
      2015.11.03 14:29:48 INFO   es[o.elasticsearch.node]  [sonar-1446557344237] stopping ...
      2015.11.03 14:29:48 WARN   es[o.e.indices.cluster]  [sonar-1446557344237] [[issues][6]] marking and sending shard failed due to [failed recovery]
      org.elasticsearch.indices.recovery.RecoveryFailedException: [issues][6]: Recovery failed from [sonar-1446557349467][VNgtzoCzSQ-qVuQspkkhyg][qaf-sonarqube52-nodees-1.internal.sonarsource.com][inet[qaf-sonarqube52-nodees-1.internal.sonarsource.com/10.150.20.71:9001]]{rack_id=sonar-1446557349467, master=false} into [sonar-1446557344237][MP-8k1_8SbOpOXHoaWG9AA][qaf-sonarqube52.internal.sonarsource.com][inet[qaf-sonarqube52.internal.sonarsource.com/10.150.20.131:9001]]{rack_id=sonar-1446557344237, master=true}
              at org.elasticsearch.indices.recovery.RecoveryTarget.doRecovery(RecoveryTarget.java:280) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.indices.recovery.RecoveryTarget.access$700(RecoveryTarget.java:70) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.indices.recovery.RecoveryTarget$RecoveryRunner.doRun(RecoveryTarget.java:567) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36) [elasticsearch-1.7.2.jar:na]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_65]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_65]
              at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
      Caused by: org.elasticsearch.transport.TransportException: transport stopped, action: internal:index/shard/recovery/start_recovery
              at org.elasticsearch.transport.TransportService$2.run(TransportService.java:178) ~[elasticsearch-1.7.2.jar:na]
              ... 3 common frames omitted
      2015.11.03 14:29:48 WARN   es[o.e.n.c.DefaultChannelPipeline]  An exception was thrown by an exception handler.
      java.util.concurrent.RejectedExecutionException: Worker has already been shutdown
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.registerTask(AbstractNioSelector.java:120) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:72) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:56) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioChannelSink.execute(AbstractNioChannelSink.java:34) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.execute(DefaultChannelPipeline.java:636) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.Channels.fireExceptionCaughtLater(Channels.java:496) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:46) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.notifyHandlerException(DefaultChannelPipeline.java:658) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:577) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.Channels.write(Channels.java:704) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.Channels.write(Channels.java:671) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.AbstractChannel.write(AbstractChannel.java:348) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.transport.netty.NettyTransportChannel.sendResponse(NettyTransportChannel.java:105) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.transport.netty.NettyTransportChannel.sendResponse(NettyTransportChannel.java:76) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.indices.recovery.RecoveryTarget$FileChunkTransportRequestHandler.messageReceived(RecoveryTarget.java:532) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.indices.recovery.RecoveryTarget$FileChunkTransportRequestHandler.messageReceived(RecoveryTarget.java:468) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.doRun(MessageChannelHandler.java:279) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36) [elasticsearch-1.7.2.jar:na]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_65]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_65]
              at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
      2015.11.03 14:29:48 WARN   es[o.e.c.action.shard]  [sonar-1446557344237] [issues][6] received shard failed for [issues][6], node[MP-8k1_8SbOpOXHoaWG9AA], [R], s[INITIALIZING], unassigned_info[[reason=CLUSTER_RECOVERED], at[2015-11-03T13:29:11.000Z]], indexUUID [tGjN1f72RSqOBzDnj4dflw], reason [shard failure [failed recovery][RecoveryFailedException[[issues][6]: Recovery failed from [sonar-1446557349467][VNgtzoCzSQ-qVuQspkkhyg][qaf-sonarqube52-nodees-1.internal.sonarsource.com][inet[qaf-sonarqube52-nodees-1.internal.sonarsource.com/10.150.20.71:9001]]{rack_id=sonar-1446557349467, master=false} into [sonar-1446557344237][MP-8k1_8SbOpOXHoaWG9AA][qaf-sonarqube52.internal.sonarsource.com][inet[qaf-sonarqube52.internal.sonarsource.com/10.150.20.131:9001]]{rack_id=sonar-1446557344237, master=true}]; nested: TransportException[transport stopped, action: internal:index/shard/recovery/start_recovery]; ]]
      2015.11.03 14:29:48 WARN   es[o.e.n.c.DefaultChannelPipeline]  An exception was thrown by an exception handler.
      java.util.concurrent.RejectedExecutionException: Worker has already been shutdown
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.registerTask(AbstractNioSelector.java:120) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:72) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:56) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioChannelSink.execute(AbstractNioChannelSink.java:34) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.execute(DefaultChannelPipeline.java:636) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.Channels.fireExceptionCaughtLater(Channels.java:496) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:46) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.notifyHandlerException(DefaultChannelPipeline.java:658) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:577) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.Channels.write(Channels.java:704) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.Channels.write(Channels.java:671) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.AbstractChannel.write(AbstractChannel.java:348) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.transport.netty.NettyTransportChannel.sendResponse(NettyTransportChannel.java:105) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.transport.netty.NettyTransportChannel.sendResponse(NettyTransportChannel.java:76) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.indices.recovery.RecoveryTarget$FileChunkTransportRequestHandler.messageReceived(RecoveryTarget.java:532) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.indices.recovery.RecoveryTarget$FileChunkTransportRequestHandler.messageReceived(RecoveryTarget.java:468) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.doRun(MessageChannelHandler.java:279) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36) [elasticsearch-1.7.2.jar:na]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_65]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_65]
              at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
      2015.11.03 14:29:49 WARN   es[o.e.n.c.DefaultChannelPipeline]  An exception was thrown by an exception handler.
      java.util.concurrent.RejectedExecutionException: Worker has already been shutdown
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.registerTask(AbstractNioSelector.java:120) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:72) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:56) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioChannelSink.execute(AbstractNioChannelSink.java:34) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.execute(DefaultChannelPipeline.java:636) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.Channels.fireExceptionCaughtLater(Channels.java:496) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:46) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.notifyHandlerException(DefaultChannelPipeline.java:658) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:577) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.Channels.write(Channels.java:704) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.Channels.write(Channels.java:671) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.AbstractChannel.write(AbstractChannel.java:348) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.transport.netty.NettyTransportChannel.sendResponse(NettyTransportChannel.java:105) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.transport.netty.NettyTransportChannel.sendResponse(NettyTransportChannel.java:76) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.indices.recovery.RecoveryTarget$FileChunkTransportRequestHandler.messageReceived(RecoveryTarget.java:532) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.indices.recovery.RecoveryTarget$FileChunkTransportRequestHandler.messageReceived(RecoveryTarget.java:468) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.doRun(MessageChannelHandler.java:279) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36) [elasticsearch-1.7.2.jar:na]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_65]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_65]
              at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
      2015.11.03 14:29:49 WARN   es[o.e.indices.cluster]  [sonar-1446557344237] [[issues][2]] marking and sending shard failed due to [failed recovery]
      org.elasticsearch.indices.recovery.RecoveryFailedException: [issues][2]: Recovery failed from [sonar-1446557356479][HCw-52JXQzGIU3tPU9UwUw][qaf-sonarqube52-nodees-2.internal.sonarsource.com][inet[qaf-sonarqube52-nodees-2.internal.sonarsource.com/10.150.20.75:9001]]{rack_id=sonar-1446557356479, master=false} into [sonar-1446557344237][MP-8k1_8SbOpOXHoaWG9AA][qaf-sonarqube52.internal.sonarsource.com][inet[qaf-sonarqube52.internal.sonarsource.com/10.150.20.131:9001]]{rack_id=sonar-1446557344237, master=true}
              at org.elasticsearch.indices.recovery.RecoveryTarget.doRecovery(RecoveryTarget.java:280) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.indices.recovery.RecoveryTarget.access$700(RecoveryTarget.java:70) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.indices.recovery.RecoveryTarget$RecoveryRunner.doRun(RecoveryTarget.java:567) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36) [elasticsearch-1.7.2.jar:na]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_65]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_65]
              at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
      Caused by: org.elasticsearch.transport.TransportException: transport stopped, action: internal:index/shard/recovery/start_recovery
              at org.elasticsearch.transport.TransportService$2.run(TransportService.java:178) ~[elasticsearch-1.7.2.jar:na]
              ... 3 common frames omitted
      2015.11.03 14:29:49 WARN   es[o.e.c.action.shard]  [sonar-1446557344237] [issues][2] received shard failed for [issues][2], node[MP-8k1_8SbOpOXHoaWG9AA], [R], s[INITIALIZING], unassigned_info[[reason=CLUSTER_RECOVERED], at[2015-11-03T13:29:11.000Z]], indexUUID [tGjN1f72RSqOBzDnj4dflw], reason [shard failure [failed recovery][RecoveryFailedException[[issues][2]: Recovery failed from [sonar-1446557356479][HCw-52JXQzGIU3tPU9UwUw][qaf-sonarqube52-nodees-2.internal.sonarsource.com][inet[qaf-sonarqube52-nodees-2.internal.sonarsource.com/10.150.20.75:9001]]{rack_id=sonar-1446557356479, master=false} into [sonar-1446557344237][MP-8k1_8SbOpOXHoaWG9AA][qaf-sonarqube52.internal.sonarsource.com][inet[qaf-sonarqube52.internal.sonarsource.com/10.150.20.131:9001]]{rack_id=sonar-1446557344237, master=true}]; nested: TransportException[transport stopped, action: internal:index/shard/recovery/start_recovery]; ]]
      2015.11.03 14:29:49 INFO   es[o.elasticsearch.node]  [sonar-1446557344237] stopped
      2015.11.03 14:29:49 INFO   es[o.elasticsearch.node]  [sonar-1446557344237] closing ...
      2015.11.03 14:29:49 WARN   es[o.e.n.c.DefaultChannelPipeline]  An exception was thrown by an exception handler.
      java.util.concurrent.RejectedExecutionException: Worker has already been shutdown
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.registerTask(AbstractNioSelector.java:120) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:72) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:56) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioChannelSink.execute(AbstractNioChannelSink.java:34) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.execute(DefaultChannelPipeline.java:636) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.Channels.fireExceptionCaughtLater(Channels.java:496) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:46) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.notifyHandlerException(DefaultChannelPipeline.java:658) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:577) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.Channels.write(Channels.java:704) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.Channels.write(Channels.java:671) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.AbstractChannel.write(AbstractChannel.java:348) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.transport.netty.NettyTransportChannel.sendResponse(NettyTransportChannel.java:105) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.transport.netty.NettyTransportChannel.sendResponse(NettyTransportChannel.java:76) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.indices.recovery.RecoveryTarget$FileChunkTransportRequestHandler.messageReceived(RecoveryTarget.java:532) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.indices.recovery.RecoveryTarget$FileChunkTransportRequestHandler.messageReceived(RecoveryTarget.java:468) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.doRun(MessageChannelHandler.java:279) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36) [elasticsearch-1.7.2.jar:na]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_65]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_65]
              at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
      2015.11.03 14:29:49 WARN   es[o.e.n.c.DefaultChannelPipeline]  An exception was thrown by an exception handler.
      java.util.concurrent.RejectedExecutionException: Worker has already been shutdown
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.registerTask(AbstractNioSelector.java:120) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:72) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:56) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioChannelSink.execute(AbstractNioChannelSink.java:34) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.execute(DefaultChannelPipeline.java:636) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.Channels.fireExceptionCaughtLater(Channels.java:496) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:46) ~[elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.notifyHandlerException(DefaultChannelPipeline.java:658) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:577) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.Channels.write(Channels.java:704) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.Channels.write(Channels.java:671) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.netty.channel.AbstractChannel.write(AbstractChannel.java:348) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.transport.netty.NettyTransportChannel.sendResponse(NettyTransportChannel.java:105) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.transport.netty.NettyTransportChannel.sendResponse(NettyTransportChannel.java:76) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.indices.recovery.RecoveryTarget$FileChunkTransportRequestHandler.messageReceived(RecoveryTarget.java:532) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.indices.recovery.RecoveryTarget$FileChunkTransportRequestHandler.messageReceived(RecoveryTarget.java:468) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.doRun(MessageChannelHandler.java:279) [elasticsearch-1.7.2.jar:na]
              at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36) [elasticsearch-1.7.2.jar:na]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_65]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_65]
              at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
      2015.11.03 14:29:49 INFO   es[o.elasticsearch.node]  [sonar-1446557344237] closed
      2015.11.03 14:29:49 INFO  app[o.s.p.m.TerminatorThread] Process[search] is stopped
      <-- Wrapper Stopped
      

        Attachments

          Activity

            People

            Assignee:
            sebastien.lesaint Sebastien Lesaint
            Reporter:
            simon.brandhof Simon Brandhof (Inactive)
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Due:
              Created:
              Updated:
              Resolved: