Uploaded image for project: 'SonarQube'
  1. SonarQube
  2. SONAR-11467

Deadlock on project and portfolio analysis with multiple workers on MSSQL

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 7.5
    • Component/s: Compute Engine
    • Labels:
    • Edition:
      Enterprise
    • Production Notes:
      None

      Description

      When multiple workers are activated, deadlocks frequently occur during project and portofolio analysis.

      Reproduced with MSSQL.

      Portfolio calculation:

      org.apache.ibatis.exceptions.PersistenceException: 
      ### Error updating database.  Cause: com.microsoft.sqlserver.jdbc.SQLServerException: Transaction (Process ID 109) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
      ### The error may involve org.sonar.db.measure.LiveMeasureMapper.deleteByProjectUuidExcludingMarker-Inline
      ### The error occurred while setting parameters
      ### SQL: delete from live_measures     where     project_uuid = ? and     (update_marker != ? or update_marker is null)
      ### Cause: com.microsoft.sqlserver.jdbc.SQLServerException: Transaction (Process ID 109) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
               at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)
               at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:200)
               at org.apache.ibatis.session.defaults.DefaultSqlSession.delete(DefaultSqlSession.java:213)
               at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:68)
               at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:59)
               at com.sun.proxy.$Proxy45.deleteByProjectUuidExcludingMarker(Unknown Source)
               at org.sonar.db.measure.LiveMeasureDao.deleteByProjectUuidExcludingMarker(LiveMeasureDao.java:112)
               at org.sonar.ce.task.projectanalysis.step.PersistLiveMeasuresStep.execute(PersistLiveMeasuresStep.java:92)
               at org.sonar.ce.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:69)
               at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:55)
               at com.A.B.C.E.E.A.B.A(Unknown Source)
               at com.A.B.C.E.E.A.D.A(Unknown Source)
               at com.A.B.C.G.A.process(Unknown Source)
               at org.sonar.ce.taskprocessor.CeWorkerImpl.executeTask(CeWorkerImpl.java:136)
               at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:99)
               at org.sonar.ce.taskprocessor.CeWorkerImpl.withCustomizedThreadName(CeWorkerImpl.java:83)
               at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:75)
               at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:45)
               at java.util.concurrent.FutureTask.run(Unknown Source)
               at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
               at java.util.concurrent.FutureTask.run(Unknown Source)
               at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source)
               at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
               at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
               at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
               at java.lang.Thread.run(Unknown Source)
      Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Transaction (Process ID 109) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
               at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:258)
               at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1535)
               at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:467)
               at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:409)
               at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7151)
               at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2478)
               at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:219)
               at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:199)
               at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.execute(SQLServerPreparedStatement.java:393)
               at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
               at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
               at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:46)
               at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:74)
               at org.apache.ibatis.executor.ReuseExecutor.doUpdate(ReuseExecutor.java:52)
               at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117)
               at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:76)
               at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:198)
      

      Project analysis:

      org.sonar.ce.task.projectanalysis.component.VisitException: Visit of Component {key=XXX,uuid=YYY,type=DIRECTORY} failed
               at org.sonar.ce.task.projectanalysis.component.VisitException.rethrowOrWrap(VisitException.java:44)
               at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visit(DepthTraversalTypeAwareCrawler.java:41)
               at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitChildren(DepthTraversalTypeAwareCrawler.java:98)
               at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitImpl(DepthTraversalTypeAwareCrawler.java:54)
               at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visit(DepthTraversalTypeAwareCrawler.java:39)
               at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitChildren(DepthTraversalTypeAwareCrawler.java:98)
               at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitImpl(DepthTraversalTypeAwareCrawler.java:54)
               at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visit(DepthTraversalTypeAwareCrawler.java:39)
               at org.sonar.ce.task.projectanalysis.step.PersistLiveMeasuresStep.execute(PersistLiveMeasuresStep.java:91)
               at org.sonar.ce.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:69)
               at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:55)
               at org.sonar.ce.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:81)
               at org.sonar.ce.taskprocessor.CeWorkerImpl.executeTask(CeWorkerImpl.java:136)
               at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:99)
               at org.sonar.ce.taskprocessor.CeWorkerImpl.withCustomizedThreadName(CeWorkerImpl.java:83)
               at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:75)
               at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:45)
               at java.util.concurrent.FutureTask.run(Unknown Source)
               at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
               at java.util.concurrent.FutureTask.run(Unknown Source)
               at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source)
               at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
               at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
               at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
               at java.lang.Thread.run(Unknown Source)
      Caused by: org.apache.ibatis.exceptions.PersistenceException: 
      ### Error updating database.  Cause: com.microsoft.sqlserver.jdbc.SQLServerException: Transaction (Process ID 109) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
      ### The error may involve org.sonar.db.measure.LiveMeasureMapper.update-Inline
      ### The error occurred while setting parameters
      ### Cause: com.microsoft.sqlserver.jdbc.SQLServerException: Transaction (Process ID 109) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
               at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)
               at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:200)
               at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:63)
               at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:59)
               at com.sun.proxy.$Proxy45.update(Unknown Source)
               at org.sonar.db.measure.LiveMeasureDao.insertOrUpdate(LiveMeasureDao.java:103)
               at org.sonar.ce.task.projectanalysis.step.PersistLiveMeasuresStep$MeasureVisitor.visitAny(PersistLiveMeasuresStep.java:129)
               at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitNode(DepthTraversalTypeAwareCrawler.java:67)
               at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitImpl(DepthTraversalTypeAwareCrawler.java:51)
               at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visit(DepthTraversalTypeAwareCrawler.java:39)
               ... 23 more
      Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Transaction (Process ID 109) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
               at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:258)
               at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1535)
               at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:467)
               at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:409)
               at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7151)
               at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2478)
               at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:219)
               at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:199)
               at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.execute(SQLServerPreparedStatement.java:393)
               at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
               at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
               at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:46)
               at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:74)
               at org.apache.ibatis.executor.ReuseExecutor.doUpdate(ReuseExecutor.java:52)
               at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117)
               at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:76)
               at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:198)
      

        Issue Links

          Activity

          Hide
          sebastien.lesaint Sebastien Lesaint added a comment -

          "commit after each component" has been implemented and resolves this ticket

          Show
          sebastien.lesaint Sebastien Lesaint added a comment - "commit after each component" has been implemented and resolves this ticket
          Hide
          schwarzmann Marcel Schwarzmann added a comment -

          Will this also be included in the next LTS 6.7.x update?

          Show
          schwarzmann Marcel Schwarzmann added a comment - Will this also be included in the next LTS 6.7.x update?
          Hide
          christophe.levis Christophe Levis added a comment -

          Marcel Schwarzmann We are not aware of such a problem on the LTS. If you face an issue, please report it to the support team or on the community forum.

          Show
          christophe.levis Christophe Levis added a comment - Marcel Schwarzmann We are not aware of such a problem on the LTS. If you face an issue, please report it to the support team or on the community forum .

            People

            • Assignee:
              sebastien.lesaint Sebastien Lesaint
              Reporter:
              christophe.levis Christophe Levis
            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Due:
                Created:
                Updated:
                Resolved: