Uploaded image for project: 'SonarHTML'
  1. SonarHTML
  2. SONARHTML-79

Fix tokenizing for files containing BOM

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Metric, Parsing
    • Labels:
      None

      Description

      When analyzing file containing BOM

      WARN: Giving up highlighting/handling duplication for file bar.html
      java.lang.IllegalArgumentException: Unable to register token in file bar.html
      	at org.sonar.api.batch.sensor.cpd.internal.DefaultCpdTokens.addToken(DefaultCpdTokens.java:79)
      	at org.sonar.plugins.web.core.WebTokensVisitor.highlightAndDuplicate(WebTokensVisitor.java:67)
      	at org.sonar.plugins.web.core.WebTokensVisitor.startDocument(WebTokensVisitor.java:49)
      	at org.sonar.plugins.web.visitor.HtmlAstScanner.scan(HtmlAstScanner.java:75)
      	at org.sonar.plugins.web.visitor.HtmlAstScanner.scan(HtmlAstScanner.java:59)
      	at org.sonar.plugins.web.core.WebSensor.execute(WebSensor.java:98)
      	at org.sonar.scanner.sensor.SensorWrapper.analyse(SensorWrapper.java:53)
      	at org.sonar.scanner.phases.SensorsExecutor.executeSensor(SensorsExecutor.java:88)
      	at org.sonar.scanner.phases.SensorsExecutor.execute(SensorsExecutor.java:82)
      	at org.sonar.scanner.phases.SensorsExecutor.execute(SensorsExecutor.java:68)
      	at org.sonar.scanner.phases.AbstractPhaseExecutor.execute(AbstractPhaseExecutor.java:88)
      	at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:182)
      	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
      	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
      	at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:312)
      	at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:307)
      	at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:281)
      	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
      	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
      	at org.sonar.scanner.task.ScanTask.execute(ScanTask.java:48)
      	at org.sonar.scanner.task.TaskContainer.doAfterStart(TaskContainer.java:81)
      	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
      	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
      	at org.sonar.scanner.bootstrap.GlobalContainer.executeTask(GlobalContainer.java:132)
      	at org.sonar.batch.bootstrapper.Batch.doExecuteTask(Batch.java:116)
      	at org.sonar.batch.bootstrapper.Batch.executeTask(Batch.java:111)
      	at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:63)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
      	at com.sun.proxy.$Proxy0.execute(Unknown Source)
      	at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:233)
      	at org.sonarsource.scanner.api.EmbeddedScanner.runAnalysis(EmbeddedScanner.java:151)
      	at org.sonarsource.scanner.cli.Main.runAnalysis(Main.java:123)
      	at org.sonarsource.scanner.cli.Main.execute(Main.java:77)
      	at org.sonarsource.scanner.cli.Main.main(Main.java:61)
      Caused by: java.lang.IllegalArgumentException: 7 is not a valid line offset for pointer. File bar.html has 6 character(s) at line 1
      	at org.sonar.api.internal.google.common.base.Preconditions.checkArgument(Preconditions.java:145)
      	at org.sonar.api.batch.fs.internal.DefaultInputFile.checkValid(DefaultInputFile.java:312)
      	at org.sonar.api.batch.fs.internal.DefaultInputFile.newPointer(DefaultInputFile.java:246)
      	at org.sonar.api.batch.fs.internal.DefaultInputFile.newRange(DefaultInputFile.java:262)
      	at org.sonar.api.batch.sensor.cpd.internal.DefaultCpdTokens.addToken(DefaultCpdTokens.java:77)
      	... 37 common frames omitted
      
      

      To fix the issue we should stop reading physical file:
      https://github.com/SonarSource/sonar-web/blob/master/sonar-web-plugin/src/main/java/org/sonar/plugins/web/core/WebSensor.java#L97

      And instead use InputFile#contents()

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                elena.vilchik Elena Vilchik
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: