Uploaded image for project: 'SonarQube'
  1. SonarQube
  2. SONAR-12677

Remove private dependencies from build.gradle

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 8.1
    • Component/s: None
    • Labels:
      None
    • Edition:
      Community
    • Production Notes:
      None

      Description

      WHY

      Since 8.0 and commit of SONAR-12384, SQ can no longer be built from source available from public repository.

      The build fails with an error such as the following:

      FAILURE: Build failed with an exception.
      
      * Where:
      Build file '/home/travis/build/SonarSource/sonarqube.ori/server/sonar-docs/build.gradle' line: 51
      
      * What went wrong:
      Execution failed for task ':server:sonar-docs:extractAnalyzerDocFiles'.
      > Could not resolve all files for configuration ':server:sonar-docs:languagePlugin'.
         > Could not resolve com.sonarsource.abap:sonar-abap-plugin:3.8.0.2034.
           Required by:
               project :server:sonar-docs
            > Could not resolve com.sonarsource.abap:sonar-abap-plugin:3.8.0.2034.
               > Could not get resource 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/abap/sonar-abap-plugin/3.8.0.2034/sonar-abap-plugin-3.8.0.2034.pom'.
                  > Could not GET 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/abap/sonar-abap-plugin/3.8.0.2034/sonar-abap-plugin-3.8.0.2034.pom'. Received status code 401 from server: Unauthorized
         > Could not resolve com.sonarsource.slang:sonar-apex-plugin:1.7.0.883.
           Required by:
               project :server:sonar-docs
            > Could not resolve com.sonarsource.slang:sonar-apex-plugin:1.7.0.883.
               > Could not get resource 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/slang/sonar-apex-plugin/1.7.0.883/sonar-apex-plugin-1.7.0.883.pom'.
                  > Could not GET 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/slang/sonar-apex-plugin/1.7.0.883/sonar-apex-plugin-1.7.0.883.pom'. Received status code 401 from server: Unauthorized
         > Could not resolve com.sonarsource.cpp:sonar-cfamily-plugin:6.4.0.11646.
           Required by:
               project :server:sonar-docs
            > Could not resolve com.sonarsource.cpp:sonar-cfamily-plugin:6.4.0.11646.
               > Could not get resource 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/cpp/sonar-cfamily-plugin/6.4.0.11646/sonar-cfamily-plugin-6.4.0.11646.pom'.
                  > Could not GET 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/cpp/sonar-cfamily-plugin/6.4.0.11646/sonar-cfamily-plugin-6.4.0.11646.pom'. Received status code 401 from server: Unauthorized
         > Could not resolve com.sonarsource.cobol:sonar-cobol-plugin:4.4.0.3403.
           Required by:
               project :server:sonar-docs
            > Could not resolve com.sonarsource.cobol:sonar-cobol-plugin:4.4.0.3403.
               > Could not get resource 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/cobol/sonar-cobol-plugin/4.4.0.3403/sonar-cobol-plugin-4.4.0.3403.pom'.
                  > Could not GET 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/cobol/sonar-cobol-plugin/4.4.0.3403/sonar-cobol-plugin-4.4.0.3403.pom'. Received status code 401 from server: Unauthorized
         > Could not resolve com.sonarsource.pli:sonar-pli-plugin:1.10.0.1880.
           Required by:
               project :server:sonar-docs
            > Could not resolve com.sonarsource.pli:sonar-pli-plugin:1.10.0.1880.
               > Could not get resource 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/pli/sonar-pli-plugin/1.10.0.1880/sonar-pli-plugin-1.10.0.1880.pom'.
                  > Could not GET 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/pli/sonar-pli-plugin/1.10.0.1880/sonar-pli-plugin-1.10.0.1880.pom'. Received status code 401 from server: Unauthorized
         > Could not resolve com.sonarsource.plsql:sonar-plsql-plugin:3.4.1.2576.
           Required by:
               project :server:sonar-docs
            > Could not resolve com.sonarsource.plsql:sonar-plsql-plugin:3.4.1.2576.
               > Could not get resource 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/plsql/sonar-plsql-plugin/3.4.1.2576/sonar-plsql-plugin-3.4.1.2576.pom'.
                  > Could not GET 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/plsql/sonar-plsql-plugin/3.4.1.2576/sonar-plsql-plugin-3.4.1.2576.pom'. Received status code 401 from server: Unauthorized
         > Could not resolve com.sonarsource.rpg:sonar-rpg-plugin:2.3.0.1187.
           Required by:
               project :server:sonar-docs
            > Could not resolve com.sonarsource.rpg:sonar-rpg-plugin:2.3.0.1187.
               > Could not get resource 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/rpg/sonar-rpg-plugin/2.3.0.1187/sonar-rpg-plugin-2.3.0.1187.pom'.
                  > Could not GET 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/rpg/sonar-rpg-plugin/2.3.0.1187/sonar-rpg-plugin-2.3.0.1187.pom'. Received status code 401 from server: Unauthorized
         > Could not resolve com.sonarsource.swift:sonar-swift-plugin:4.1.0.3087.
           Required by:
               project :server:sonar-docs
            > Could not resolve com.sonarsource.swift:sonar-swift-plugin:4.1.0.3087.
               > Could not get resource 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/swift/sonar-swift-plugin/4.1.0.3087/sonar-swift-plugin-4.1.0.3087.pom'.
                  > Could not GET 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/swift/sonar-swift-plugin/4.1.0.3087/sonar-swift-plugin-4.1.0.3087.pom'. Received status code 401 from server: Unauthorized
         > Could not resolve com.sonarsource.tsql:sonar-tsql-plugin:1.4.0.3334.
           Required by:
               project :server:sonar-docs
            > Could not resolve com.sonarsource.tsql:sonar-tsql-plugin:1.4.0.3334.
               > Could not get resource 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/tsql/sonar-tsql-plugin/1.4.0.3334/sonar-tsql-plugin-1.4.0.3334.pom'.
                  > Could not GET 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/tsql/sonar-tsql-plugin/1.4.0.3334/sonar-tsql-plugin-1.4.0.3334.pom'. Received status code 401 from server: Unauthorized
         > Could not resolve com.sonarsource.plugins.vb:sonar-vb-plugin:2.6.0.1875.
           Required by:
               project :server:sonar-docs
            > Could not resolve com.sonarsource.plugins.vb:sonar-vb-plugin:2.6.0.1875.
               > Could not get resource 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/plugins/vb/sonar-vb-plugin/2.6.0.1875/sonar-vb-plugin-2.6.0.1875.pom'.
                  > Could not GET 'https://repox.jfrog.io/repox/sonarsource/com/sonarsource/plugins/vb/sonar-vb-plugin/2.6.0.1875/sonar-vb-plugin-2.6.0.1875.pom'. Received status code 401 from server: Unauthorized 

      This failure comes from the fact that module sonar-docs defines dependencies on private plugins.

      It is essential that SonarQube can be built from sources available from the public repository.

      Note: this problem affects tag 8.0 in public repository, which implies that 8.0 can't be built from source

      WHAT

      Module sonar-docs has two purpose:

      1. it holds the markdown files (in sources) for embedded documentation
      2. it produces an artifact used for the static documentation available at docs.sonarqube.org (made from the same markdown)

       Currently, both static documentation and the embedded one provide the documentation for all plugins, whatever the edition the documentation is embedded into.

      The error above highlights the fact that this is problem for the Community edition and to fix it, we will simply end it:

      • the static documentation should display the documentation of all plugins
      • the embedded documentation should display only the documentation of the plugins available in the current edition (ie. the installed plugins)

      Not rendering the doc if plugin is not installed

      Documentation contains a hard coded menu which lists the plugins.

      Currently, when a plugin is not installed and/or does not provide its own documentation, the documentation for this plugin which is SQ repository is displayed. This fallback mechanism exists because not all plugin provide their own documentation yet.

      This implies that even if plugin is not part of the current edition, some documentation for it will be displayed.

      That would be convenient because it makes the hard coded menu a no-problem.

      However, in the case a plugin does provide its own documentation but is not part of the current edition, displaying the one from the SQ repository would mean displaying an out-of-date documentation.

       If this is not OK, instead, the doc menu should be updated to hide entries at runtime for plugins which are not installed (information available from api/plugins/installed) => plugins shipping with their own edition are rare which means the doc in SQ is up to date. Also, 8.0 is a short lived version. This problem exists but will be taken care of separately (see SONAR-12681)

      Not producing static doc in public repository

      Static doc contains the doc of all plugins and is all generated at build time.

      Private plugins are not available in the public repository, which implies that static doc can't be generated with the embedded doc these plugins provide. Which means the artifact produced from the public repository sources may contain outdated data.

       does it make sense to produce a public static doc artifact which is only intended to be used for https://docs.sonarqube.org? => it doesn't, artifact should not be produced

       if so, is it ok to produce one with outdated information? => see above

       if not, then we might just as well not produce this artifact at all in the public repository => yes

      HOW

      No static doc artifact in public repository

      Module sonar-doc will not be included by settings.gradle at the root of the project, but by settings.gradle in the private directory.

      This will both make static documentation artifacts not available from public sources and fix the bug above as gradle won't attempt to resolve private artifacts used by this module at all.

      (nice-to-have) Improvement

      The list of bundled plugins is used by multiple modules and currently duplicated:

      • each edition module
      • sonar-docs

      The solution to this ticket is to create Gradle configurations in the root build.gradle (for the community edition) and the build.gradle in the private directory (for commercial editions) with the list of plugins in each edition.

      All the modules listed above will used these configuration, which will remove duplication.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                sebastien.lesaint Sebastien Lesaint
                Reporter:
                wouter.admiraal Wouter Admiraal
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Due:
                  Created:
                  Updated:
                  Resolved: