Uploaded image for project: 'Product Roadmaps'
  1. Product Roadmaps
  2. MMF-885

More powerful scanner configuration "protocol"

    XMLWordPrintable

    Details

    • Type: MMF
    • Status: Closed
    • Priority: Major
    • Resolution: Won't Do
    • Labels:

      Description

      The way to pass configuration between scanners and scanner-engine is to use key/value strings. Also, this concept "leaks" into end user experience:

      • Scanner CLI use a Java properties file + -Dkey=value on command line
      • Maven support <properties> in pom.xml + -Dkey=value on command line
      • Gradle support properties block + -Dkey=value on command line
      • MSBuild support adding entries in SonarQube.Analysis.xml + /d:key=value on command line

      Limits of this key/value syntax

      Multi-values properties

      Some properties are known to accept multi-values. The syntax is to separate values with a comma. There is no escape mechanism, so it prevents any value to contain a comma. Note that it really depends on the consumer: if the consumer expects a multi-value property, then value is split on comma. But if consumer expects a single value, then no split attempt is made, and so the value can contain a comma. SONAR-9198

      Module Inheritance

      Syntax for modules is:

      sonar.modules=moduleid1,moduleid2,...
      sonar.sources=src/main/java
      
      moduleid1.sonar.sources=src/java
      
      moduleis1.sonar.modules=moduleid1_1,moduleid1_2,...
      moduleid1.moduleid1_1.sonar.sources=src/main/js
      

      Concerns:

      • complexity/size of properties (especially when there are many levels). This makes investigating harder, was a source of numerous bugs (but under control now) and also a bit costly to process (but not that important)
      • a property declared in a parent module is inherited in children modules. There is no way to declare a property only for a module, and prevent it to be inherited. On the other side, there is no way to declare a property to be inherited but not "applied" to the current module.
      • it is difficult for a user to apply a property to a specific module from the command line - at least with maven, it's impossible.

      Limits of the SonarQube Scanner CLI configuration file (sonar-project.properties)

      Encoding

      The file is read with ISO-8859-1 encoding. This makes it difficult to use it in non european languages (sonar.projectName, sonar.projectDescription, ...)
      SUPPORT-3160

      Backslash escape

      Backslash is used to escape some characters. It means that to properly pass Windows file paths, users have to double the backslash. SQSCANNER-16

      sonar.jacoco.reportPaths=target\\jacoco.exec
      

      Multiline

      Multiline syntax is not user friendly, especially when using multi-value properties

      sonar.sources=folder1,\
           folder2,\
           folder3
      

      Proposal

      There are 2 independent problems to solve:

      1. the technical protocol between scanners and scanner-engine
      2. the format of the configuration file for the SonarQube Scanner for CLI

      Scanner client protocol

      The only real issue is the support of multi-value properties when the value contains a comma. Other concerns are minor (even inheritance) and the complexity is now under control. It has also proven to help evolving.

      Starting from SonarQube 6.5 we could support a new syntax to pass multi-value properties. We could for example support comma escaping.

      sonar.sources=c:\a\path\with\,comma

      The problem is that this is a breaking change.

      sonar.sources=src1\,src2\
      

      would have to be changed to:

      sonar.sources=src1\\,src2\
      

      It is also a bit ugly to have \ only escaping comma. Maybe it would be simpler/more robust to adopt an established syntax, for example JSON string array:

      sonar.sources=["src1\\", "src2\\"]

      It is unlikely to have existing multi-value properties following this syntax. So backward compatibility could be easy to achieve.

      Scanner CLI configuration file

      We should not break backward compatibility. So the current format of sonar-project.property can't be changed. We can:

      • introduce a new format in a new filename (e.g. sonar-project.json)
      • keep the same name, but ask users to add a header to unlock new syntax (e.g. #format 2)
        New format should:
      • be encoded in UTF-8
      • don't require to escape backslashes (Windows paths)
      • have a simple support of multi-value properties, possibly with comma inside values, and spanning multiple lines
      • simplify multi-module configuration

      Example of using JSON:

      {
      # We can automatically append the "sonar." prefix
        "projectKey": "myproject",
        "projectName": "MyProject that support UTF-8 éù",
        "projectVersion": "${version}",
      # Another way to declare properties, without "magic" prefix
        "properties": [
          {"sonar.jacoco.reportPaths": [
              "c:\\foo.exec",
              "foo2.exec"
            ]
          },
          {"another.property": "bar"}
        ],
        "modules": [
          {
            "moduleid1": {
              "projectName": "MyModule 1"
            },
            "moduleid2": {
              "projectName": "MyModule 2"
            }
          }
        ]
      }
      

      Caveat:

      • JSON doesn't support comments. We could pre-process the file to support comments.
      • Is it really more user friendly than property file?

      Example of using an "updated" property file:

      #format 2
      sonar.projectKey=myproject
      sonar.projectName=MyProject that support UTF-8 éù
      sonar.jacoco.reportPaths=["c:\\\\foo.exec", \
          "foo2.exec"]
      

      Caveat:

      • The JSON Array syntax would need to escape backslash twice (once for property decoding, once for JSON decoding)
      • Same ugly multiline syntax

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              julien.henry Julien Henry
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: