Uploaded image for project: 'SonarQube'
  1. SonarQube
  2. SONAR-11873

Memory leak when Compute Engine persists file duplications

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 7.8
    • Labels:
      None
    • Edition:
      Community
    • Production Notes:
      None

      Description

      Details of file duplications are stored as XML documents in the DB table live_measures. All the XML documents of all project files are kept in memory before being persisted. That can make the Compute Engine task fail on out-of-memory:

      2019.03.27 17:22:58 INFO ce[AWm_8KHVS-T8uQm7MXxS][o.s.c.t.s.ComputationStepExecutor] Compute duplication data measures | status=FAILED | time=236030ms
      2019.03.27 17:29:18 ERROR ce[AWm_8KHVS-T8uQm7MXxS][o.s.c.t.CeWorkerImpl] Failed to execute task AWm_8KHVS-T8uQm7MXxS
      java.lang.OutOfMemoryError: Java heap space
       at java.util.Arrays.copyOf(Arrays.java:3332)
       at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
       at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
       at java.lang.StringBuilder.append(StringBuilder.java:136)
       at org.sonar.ce.task.projectanalysis.step.DuplicationDataMeasuresStep$DuplicationVisitor.appendDuplication(DuplicationDataMeasuresStep.java:136)
       at org.sonar.ce.task.projectanalysis.step.DuplicationDataMeasuresStep$DuplicationVisitor.appendDuplication(DuplicationDataMeasuresStep.java:128)
       at org.sonar.ce.task.projectanalysis.step.DuplicationDataMeasuresStep$DuplicationVisitor.processDuplicationBlock(DuplicationDataMeasuresStep.java:117)
       at org.sonar.ce.task.projectanalysis.step.DuplicationDataMeasuresStep$DuplicationVisitor.createXmlDuplications(DuplicationDataMeasuresStep.java:100)
       at org.sonar.ce.task.projectanalysis.step.DuplicationDataMeasuresStep$DuplicationVisitor.computeDuplications(DuplicationDataMeasuresStep.java:86)
       at org.sonar.ce.task.projectanalysis.step.DuplicationDataMeasuresStep$DuplicationVisitor.visitFile(DuplicationDataMeasuresStep.java:81)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitNode(DepthTraversalTypeAwareCrawler.java:76)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitImpl(DepthTraversalTypeAwareCrawler.java:51)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visit(DepthTraversalTypeAwareCrawler.java:39)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitChildren(DepthTraversalTypeAwareCrawler.java:95)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitImpl(DepthTraversalTypeAwareCrawler.java:54)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visit(DepthTraversalTypeAwareCrawler.java:39)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitChildren(DepthTraversalTypeAwareCrawler.java:95)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitImpl(DepthTraversalTypeAwareCrawler.java:54)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visit(DepthTraversalTypeAwareCrawler.java:39)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitChildren(DepthTraversalTypeAwareCrawler.java:95)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitImpl(DepthTraversalTypeAwareCrawler.java:54)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visit(DepthTraversalTypeAwareCrawler.java:39)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitChildren(DepthTraversalTypeAwareCrawler.java:95)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitImpl(DepthTraversalTypeAwareCrawler.java:54)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visit(DepthTraversalTypeAwareCrawler.java:39)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitChildren(DepthTraversalTypeAwareCrawler.java:95)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitImpl(DepthTraversalTypeAwareCrawler.java:54)
       at org.sonar.ce.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visit(DepthTraversalTypeAwareCrawler.java:39)
       at org.sonar.ce.task.projectanalysis.step.DuplicationDataMeasuresStep.execute(DuplicationDataMeasuresStep.java:68)
       at org.sonar.ce.task.step.ComputationStepExecutor.executeStep(ComputationStepExecutor.java:81)
       at org.sonar.ce.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:72)
       at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:59)
      2019.03.27 17:29:20 INFO ce[AWm_8KHVS-T8uQm7MXxS][o.s.c.t.CeWorkerImpl] Executed task | project=xxxxx | type=REPORT | id=AWm_8KHVS-T8uQm7MXxS | submitter=xxxxx | status=FAILED | time=733074ms

      The solution is to persist on the fly when generating XML documents.

        Attachments

          Activity

            People

            Assignee:
            simon.brandhof Simon Brandhof (Inactive)
            Reporter:
            simon.brandhof Simon Brandhof (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Due:
              Created:
              Updated:
              Resolved: