Uploaded image for project: 'SonarQube'
  1. SonarQube
  2. SONAR-12078

Compute Engine can fail with out of memory if a source file has too many lines

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.5
    • Fix Version/s: 7.8
    • Component/s: Compute Engine
    • Labels:
      None
    • Edition:
      Community
    • Production Notes:
      None

      Description

      Detection of code move inside a file was implemented in version 3.5 for issue tracking. The matrix of combinations used to match previous version of file can be huge, so algorithm is not run if matrix size reaches a given threshold. The problem is that the condition that checks the threshold is buggy. It may be incorrect in some conditions, basically when file is has more than ~47k issues.

      Examples of related out-of-memory errors:

      java.lang.OutOfMemoryError: Java heap space
          at java.util.Arrays.copyOf(Arrays.java:3181)
          at java.util.ArrayList.grow(ArrayList.java:265)
          at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:239)
          at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:231)
          at java.util.ArrayList.add(ArrayList.java:462)
          at org.sonar.core.issue.tracking.BlockRecognizer.match(BlockRecognizer.java:89)
          at org.sonar.core.issue.tracking.Tracker.detectCodeMoves(Tracker.java:67)
          at org.sonar.core.issue.tracking.Tracker.trackNonClosed(Tracker.java:43)
          at org.sonar.ce.task.projectanalysis.issue.TrackerExecution.track(TrackerExecution.java:56)
          at org.sonar.ce.task.projectanalysis.issue.IssueTrackingDelegator.track(IssueTrackingDelegator.java:53)
          at org.sonar.ce.task.projectanalysis.issue.IntegrateIssuesVisitor.visitAny(IntegrateIssuesVisitor.java:64)
          at org.sonar.ce.task.projectanalysis.component.TypeAwareVisitorWrapper.visitAny(TypeAwareVisitorWrapper.java:77)
          at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visitNode(VisitorsCrawler.java:117)
          at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visitImpl(VisitorsCrawler.java:100)
          at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visit(VisitorsCrawler.java:72)
          at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visitChildren(VisitorsCrawler.java:110)
          at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visitImpl(VisitorsCrawler.java:97)
          at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visit(VisitorsCrawler.java:72)
          at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visitChildren(VisitorsCrawler.java:110)
          at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visitImpl(VisitorsCrawler.java:97)
          at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visit(VisitorsCrawler.java:72)
          at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visitChildren(VisitorsCrawler.java:110)
          at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visitImpl(VisitorsCrawler.java:97)
          at org.sonar.ce.task.projectanalysis.component.VisitorsCrawler.visit(VisitorsCrawler.java:72)
          at org.sonar.ce.task.projectanalysis.step.ExecuteVisitorsStep.execute(ExecuteVisitorsStep.java:51)
          at org.sonar.ce.task.step.ComputationStepExecutor.executeStep(ComputationStepExecutor.java:81)
          at org.sonar.ce.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:72)
          at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:59)
          at org.sonar.ce.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:81)
          at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.executeTask(CeWorkerImpl.java:207)
          at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.run(CeWorkerImpl.java:189)
          at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:156)
      
      java.lang.IllegalStateException: Fail to select data of CE task AWo9WYhjzH1k1-Xjcr2b
          at org.sonar.db.ce.CeTaskInputDao.selectData(CeTaskInputDao.java:74)
          at org.sonar.ce.task.projectanalysis.step.ExtractReportStep.execute(ExtractReportStep.java:65)
          at org.sonar.ce.task.step.ComputationStepExecutor.executeStep(ComputationStepExecutor.java:81)
          at org.sonar.ce.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:72)
          at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:59)
          at org.sonar.ce.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:81)
          at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.executeTask(CeWorkerImpl.java:207)
          at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.run(CeWorkerImpl.java:189)
          at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:156)
          at org.sonar.ce.taskprocessor.CeWorkerImpl$TrackRunningState.get(CeWorkerImpl.java:131)
          at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:83)
          at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:51)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
      Caused by: org.postgresql.util.PSQLException: ERROR: invalid memory alloc request size 1081452017
          at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
          at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
          at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
          at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
          at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
          at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:143)
          at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:106)
          at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:122)
          at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:122)
          at org.sonar.db.ce.CeTaskInputDao.selectData(CeTaskInputDao.java:67)
          ... 19 common frames omitted
      

        Attachments

          Activity

            People

            Assignee:
            simon.brandhof Simon Brandhof (Inactive)
            Reporter:
            simon.brandhof Simon Brandhof (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Due:
              Created:
              Updated:
              Resolved: