Uploaded image for project: 'SonarQube'
  1. SonarQube
  2. SONAR-6553

Drop the Design related services and metrics

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.2
    • Component/s: DSM
    • Labels:
      None

      Description

      Facts are the following:

      • Few people are using and giving feedback on the Design features (some bugs were discovered very long after they were introduced)
      • This is not surprising because those features currently do not contribute issues nor technical debt, so developers can't manage those "flaws" (for instance, it's impossible to tell that this cycle between package is normal whereas this one is a real problem)
      • What's more, those features are mainly provided on Java projects, so SQ does not offer a consistent experience across different projects
      • Even in the Java world, not all projects benefit from those services because some (like Libraries or Dependencies) are heavily coupled with Maven
      • Features like Libraries or Dependencies do not directly relate to source code analysis, so there's no real reason why we should keep them
      • Features like the DSM can be useful (only for Java projects...), but in its current format it's unusable and most users don't understand how it can be used

      Based on all those facts, all design features should be dropped:

      • Dependencies service
      • Libraries service
      • DSM service
      • Design widgets and metrics

      This will give the opportunity to start from scratch on this topic and offer features that fully match the philosophy and the targets of the SQ platform. This is what is called "Cartography" on the SQ Roadmap page.

        Issue Links

          Activity

          fabemn OLD - Fabrice Bellingard created issue -
          fabemn OLD - Fabrice Bellingard made changes -
          Field Original Value New Value
          Description Facts are the following:
          - Few people are using and giving feedback on the Design features (some bugs are discovered very long after they were introduced)
          - This is not surprising because those features currently do not contribute issues nor technical debt, so developers can't manage those "flaws" (for instance, it's impossible to tell that this cycle between package is normal whereas this one is a real problem)
          - What's more, those features are mainly provided on Java projects, so SQ does not offer a consistent experience across different projects
          - Even in the Java world, not all projects benefit from those services because some (like Libraries or Dependencies) are heavily coupled with Maven
          - Features like Libraries or Dependencies do not directly relate to source code analysis, so there's no real reason why we should keep them
          - Features like the DSM can be useful (only for Java projects...), but in its current format it's unusable and most users don't understand how it can be used

          Based on all those facts, all design features should be dropped:
          - Dependencies service
          - Libraries service
          - DSM service
          - Design widgets and metrics

          This will give the opportunity to start from scratch on this topic and offer features that fully match the philosophy and the targets of the SQ platform. This is what is called ["Cartography" on the SQ Roadmap page|http://www.sonarqube.org/roadmap/].
          Facts are the following:
          - Few people are using and giving feedback on the Design features (some bugs were discovered very long after they were introduced)
          - This is not surprising because those features currently do not contribute issues nor technical debt, so developers can't manage those "flaws" (for instance, it's impossible to tell that this cycle between package is normal whereas this one is a real problem)
          - What's more, those features are mainly provided on Java projects, so SQ does not offer a consistent experience across different projects
          - Even in the Java world, not all projects benefit from those services because some (like Libraries or Dependencies) are heavily coupled with Maven
          - Features like Libraries or Dependencies do not directly relate to source code analysis, so there's no real reason why we should keep them
          - Features like the DSM can be useful (only for Java projects...), but in its current format it's unusable and most users don't understand how it can be used

          Based on all those facts, all design features should be dropped:
          - Dependencies service
          - Libraries service
          - DSM service
          - Design widgets and metrics

          This will give the opportunity to start from scratch on this topic and offer features that fully match the philosophy and the targets of the SQ platform. This is what is called ["Cartography" on the SQ Roadmap page|http://www.sonarqube.org/roadmap/].
          hgomez Henri Gomez made changes -
          Project Import Wed May 27 13:41:49 CEST 2015 [ 1432726909095 ]
          henri.gomez Henri Gomez made changes -
          Project Import Thu May 28 19:38:02 UTC 2015 [ 1432841882590 ]
          julien.lancelot Julien Lancelot made changes -
          Assignee Julien Lancelot [ julien.lancelot ]
          julien.lancelot Julien Lancelot made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Assignee Julien Lancelot [ julien.lancelot ] Fabrice Bellingard [ fabrice.bellingard ]
          Resolution Fixed [ 1 ]
          julien.lancelot Julien Lancelot made changes -
          Due Date 15/Jun/15
          Hide
          fabrice.bellingard Fabrice Bellingard added a comment -

          Tested.

          Show
          fabrice.bellingard Fabrice Bellingard added a comment - Tested.
          fabrice.bellingard Fabrice Bellingard made changes -
          Assignee Fabrice Bellingard [ fabrice.bellingard ] Julien Lancelot [ julien.lancelot ]
          julien.lancelot Julien Lancelot made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          freddy.mallet Freddy Mallet made changes -
          Workflow jira [ 30113 ] Default SonarSource Workflow [ 55269 ]
          fabrice.bellingard Fabrice Bellingard made changes -
          Link This issue relates to SONARJAVA-1184 [ SONARJAVA-1184 ]
          freddy.mallet Freddy Mallet made changes -
          Workflow Default SonarSource Workflow [ 55269 ] Default Agile SonarSource Workflow [ 75494 ]
          fabrice.bellingard Fabrice Bellingard made changes -
          Link This issue relates to SONAR-2269 [ SONAR-2269 ]
          alexandre.gigleux Alexandre Gigleux made changes -
          Remote Link This issue links to "Page (SonarQube)" [ 31103 ]
          fabrice.bellingard Fabrice Bellingard made changes -
          Remote Link This issue links to "Page (SonarQube)" [ 31118 ]
          fabrice.bellingard Fabrice Bellingard made changes -
          Remote Link This issue links to "Page (SonarQube)" [ 31118 ] This issue links to "Page (SonarQube)" [ 31118 ]
          fabrice.bellingard Fabrice Bellingard made changes -
          Remote Link This issue links to "Page (SonarQube)" [ 31118 ] This issue links to "Page (SonarQube)" [ 31118 ]
          fabrice.bellingard Fabrice Bellingard made changes -
          Remote Link This issue links to "Page (SonarQube)" [ 31118 ] This issue links to "Page (SonarQube)" [ 31118 ]
          freddy.mallet Freddy Mallet made changes -
          Workflow Default Agile SonarSource Workflow [ 75494 ] Default Agile SonarSource Workflow V2 [ 103983 ]
          Hide
          cardil Krzysztof Suszyński added a comment -

          I work in Centralny Ośrodek Informatyki in Poland (government IT company) and we been using this feature heavily. I can't agree with that only few people were using this feature. There are many articles on this, like:

          Additionally, squid rule: CycleBetweenPackages contribute as issue and give estimate as 1 day. I definitely can't agree that this is an improvement. We are disappointed that this nice and useful feature was dropped without previous notice.

          We still would like to use it :-/

          Even if this feature removal was part of your RoadMap, couldn't you provide a plugin to give users like us a way to use that kind of feature until new version with Cartography arrives. Also, could you give some esitimates when Cartography is planned? 6.x?

          Show
          cardil Krzysztof Suszyński added a comment - I work in Centralny Ośrodek Informatyki in Poland (government IT company) and we been using this feature heavily. I can't agree with that only few people were using this feature. There are many articles on this, like: https://dzone.com/articles/working-dependencies-eliminate http://www.sonarqube.org/fight-back-design-erosion-by-breaking-cycles-with-sonar http://stackoverflow.com/questions/34899712/what-does-it-mean-and-how-to-fix-sonarqube-java-issue-cycles-between-packages-s Additionally, squid rule: CycleBetweenPackages contribute as issue and give estimate as 1 day. I definitely can't agree that this is an improvement. We are disappointed that this nice and useful feature was dropped without previous notice. We still would like to use it :-/ Even if this feature removal was part of your RoadMap, couldn't you provide a plugin to give users like us a way to use that kind of feature until new version with Cartography arrives. Also, could you give some esitimates when Cartography is planned? 6.x?
          freddy.mallet Freddy Mallet made changes -
          Workflow Default Agile SonarSource Workflow V2 [ 103983 ] Default Agile SonarSource Workflow V3 [ 127807 ]
          Hide
          jelharou Jaafar El Harouchi added a comment -

          +1 for Krzysztof's suggestion to make it available as a separate plugin.

          I understand that deleting the code is the simplest approach, but the functionality made SonarQube analysis especially powerful. Most other open source tools struggle with DSM too and this was the best implementation so far.

          Show
          jelharou Jaafar El Harouchi added a comment - +1 for Krzysztof's suggestion to make it available as a separate plugin. I understand that deleting the code is the simplest approach, but the functionality made SonarQube analysis especially powerful. Most other open source tools struggle with DSM too and this was the best implementation so far.
          Hide
          robkam Robert Kaminski added a comment -

          I agree with Krzysztof and Jaafar and also +1 for a new plugin

          Show
          robkam Robert Kaminski added a comment - I agree with Krzysztof and Jaafar and also +1 for a new plugin
          Hide
          akieras Alessandro Kieras added a comment -

          Our company uses DSM and Pkg Tangle Index heavily in all projects.
          They provide very useful insights about the code estruture. I can't believe it was removed. Very sad. : (

          Fabrice Bellingard Julien Lancelot Is it programed to come back in any future version?

          Show
          akieras Alessandro Kieras added a comment - Our company uses DSM and Pkg Tangle Index heavily in all projects. They provide very useful insights about the code estruture. I can't believe it was removed. Very sad. : ( Fabrice Bellingard Julien Lancelot Is it programed to come back in any future version?
          Hide
          fabrice.bellingard Fabrice Bellingard added a comment -

          Alessandro Kieras This is not planned for the moment. DSM could indeed be very useful, but it was "just" a tool that you could (from time to time) use to dig and search to understand some design flaws of your code. On the other hand, everything in SonarQube must end up at some point of time in "issues" (and issues that don't produce false-positives) so that you can manage them over time and decide what to do with them. So the ideal solution would be to have something that allows to define architecture layers and that would check constraints built on top of this. But this is not in our short term road map for the moment - even though we've already discussed this a lot of time.

          Show
          fabrice.bellingard Fabrice Bellingard added a comment - Alessandro Kieras This is not planned for the moment. DSM could indeed be very useful, but it was "just" a tool that you could (from time to time) use to dig and search to understand some design flaws of your code. On the other hand, everything in SonarQube must end up at some point of time in "issues" (and issues that don't produce false-positives) so that you can manage them over time and decide what to do with them. So the ideal solution would be to have something that allows to define architecture layers and that would check constraints built on top of this. But this is not in our short term road map for the moment - even though we've already discussed this a lot of time.
          Hide
          alixwar Alix Warnke added a comment -

          +1 on keeping the functionality. We use it and find it useful to detect high-level architectural issues that we would not find otherwise.

          Show
          alixwar Alix Warnke added a comment - +1 on keeping the functionality. We use it and find it useful to detect high-level architectural issues that we would not find otherwise.
          freddy.mallet Freddy Mallet made changes -
          Workflow Default Agile SonarSource Workflow V3 [ 127807 ] Default Agile SonarSource Workflow V4 [ 155529 ]
          Hide
          Falco1001 Falco Blitz added a comment -

          +1 on keeping the functionality.
          We used it in a C++ project too detect and eleminate cyclic dependencies. All reported cycles were correct. There is no point in deleting a working feature.

          "This will give the opportunity to start from scratch on this topic and offer features that fully match ...". Fine. Drop the design related services when you have them.

          Show
          Falco1001 Falco Blitz added a comment - +1 on keeping the functionality. We used it in a C++ project too detect and eleminate cyclic dependencies. All reported cycles were correct. There is no point in deleting a working feature. "This will give the opportunity to start from scratch on this topic and offer features that fully match ...". Fine. Drop the design related services when you have them.
          freddy.mallet Freddy Mallet made changes -
          Workflow Default Agile SonarSource Workflow V4 [ 155529 ] Default Agile SonarSource Workflow V5 [ 183794 ]
          Hide
          pierre Pierre Fouché added a comment -

          + 1 on keeping the functionality.

          We used it on all our projects and having no package dependency cycle was quality rule #1. I can't believe this feature was dropped...

          Show
          pierre Pierre Fouché added a comment - + 1 on keeping the functionality. We used it on all our projects and having no package dependency cycle was quality rule #1. I can't believe this feature was dropped...
          Hide
          juraj.misur Juraj Misur added a comment -

          Hi today I was looking for package tangle index to see how is my code interlinked and ended on this page. Although I understand that it might be hard to keep it alive after switching to new analysis mode, but I don't quite understand the "facts" in description.

          Few people are using and giving feedback on the Design features (some bugs were discovered very long after they were introduced)

          Maybe the feature is usable, sufficient and stable for most of the people who use it

          This is not surprising because those features currently do not contribute issues nor technical debt, so developers can't manage those "flaws" (for instance, it's impossible to tell that this cycle between package is normal whereas this one is a real problem)

          If I don't know if particular aspect of my code is a problem or not, then I have another problem entirely... not everyone needs an issue for everything nor a technical debt metric. It's still useful as visualisation of some aspect of my code.

          What's more, those features are mainly provided on Java projects, so SQ does not offer a consistent experience across different projects

          But it might be still very usable for X% of your user base (I'd suppose mainly java)

          Even in the Java world, not all projects benefit from those services because some (like Libraries or Dependencies) are heavily coupled with Maven

          But it might be still very usable for Y% of X% of your user base (I'd suppose mainly java mainly with maven)

          Features like Libraries or Dependencies do not directly relate to source code analysis, so there's no real reason why we should keep them

          What is package tangle index if not source code analysis

          Features like the DSM can be useful (only for Java projects...), but in its current format it's unusable and most users don't understand how it can be used

          Why not improve it and educate users.

          In my case we don't use "issues" at all, just look at violations, fix couple of them, commit and forget. For some projects or teams, managing issues in sonar is unnecessary overhead. So maybe I don't understand why is it so important and gets so much space. Of course, if any kind of analysis can be translated into violation or metric, the better. But removing features because they cannot (right now) looks very short sighted, unfortunately.

          But nevertheless, I like sonar and thank you for constant improving!

          Show
          juraj.misur Juraj Misur added a comment - Hi today I was looking for package tangle index to see how is my code interlinked and ended on this page. Although I understand that it might be hard to keep it alive after switching to new analysis mode, but I don't quite understand the "facts" in description. Few people are using and giving feedback on the Design features (some bugs were discovered very long after they were introduced) Maybe the feature is usable, sufficient and stable for most of the people who use it This is not surprising because those features currently do not contribute issues nor technical debt, so developers can't manage those "flaws" (for instance, it's impossible to tell that this cycle between package is normal whereas this one is a real problem) If I don't know if particular aspect of my code is a problem or not, then I have another problem entirely... not everyone needs an issue for everything nor a technical debt metric. It's still useful as visualisation of some aspect of my code. What's more, those features are mainly provided on Java projects, so SQ does not offer a consistent experience across different projects But it might be still very usable for X% of your user base (I'd suppose mainly java) Even in the Java world, not all projects benefit from those services because some (like Libraries or Dependencies) are heavily coupled with Maven But it might be still very usable for Y% of X% of your user base (I'd suppose mainly java mainly with maven) Features like Libraries or Dependencies do not directly relate to source code analysis, so there's no real reason why we should keep them What is package tangle index if not source code analysis Features like the DSM can be useful (only for Java projects...), but in its current format it's unusable and most users don't understand how it can be used Why not improve it and educate users. In my case we don't use "issues" at all, just look at violations, fix couple of them, commit and forget. For some projects or teams, managing issues in sonar is unnecessary overhead. So maybe I don't understand why is it so important and gets so much space. Of course, if any kind of analysis can be translated into violation or metric, the better. But removing features because they cannot (right now) looks very short sighted, unfortunately. But nevertheless, I like sonar and thank you for constant improving!
          Hide
          lieflo Florian Lieb added a comment -

          +1 for keeping this functionality

          because those features currently do not contribute issues nor technical debt

          Well, there ist this issue "squid:CycleBetweenPackages" which introduces a technical dept of 1 per issue, which totaly messes the technical dept for some of our projects.

          Show
          lieflo Florian Lieb added a comment - +1 for keeping this functionality because those features currently do not contribute issues nor technical debt Well, there ist this issue "squid:CycleBetweenPackages" which introduces a technical dept of 1 per issue, which totaly messes the technical dept for some of our projects.
          Hide
          rycarpenter Ryan Carpenter added a comment -

          +1 for re-introducing or making the behavior available as a plugin

          I second most all the comments from other users here. This feature was incredibly value, and I agree with others that many of the various reasons given for removing this functioning behavior don't hold water. Seems like we're letting the perfect be the enemy of the good here. We are actually delaying upgrading our installation specifically so we don't lose this feature. I suppose at some point we'll cave, in which case we'll probably actually just stand up a separate SonarQube instance so we can refer back to the old setup when we want to look at the DSM.

          Show
          rycarpenter Ryan Carpenter added a comment - +1 for re-introducing or making the behavior available as a plugin I second most all the comments from other users here. This feature was incredibly value, and I agree with others that many of the various reasons given for removing this functioning behavior don't hold water. Seems like we're letting the perfect be the enemy of the good here. We are actually delaying upgrading our installation specifically so we don't lose this feature. I suppose at some point we'll cave, in which case we'll probably actually just stand up a separate SonarQube instance so we can refer back to the old setup when we want to look at the DSM.
          Hide
          rpatriarche Remi Patriarche added a comment -

          +1 for re-introducing this key-feature.
          Like some other users, we decided to delay our upgrade and to stay with 4.5

          Show
          rpatriarche Remi Patriarche added a comment - +1 for re-introducing this key-feature. Like some other users, we decided to delay our upgrade and to stay with 4.5
          Hide
          marks Mark Symons added a comment -

          It's the Dependencies Service that I am particularly missing.

          I use Sonatype Nexus IQ to analyse our software for security (and other) threats. This is great at telling me that there is a threat from a particular component - but not good at telling me anything about dependencies. I also tried other products, such as Black Duck Hub and White Source... none were good at providing dependency information.

          This is where SonaQube Dependencies was so useful. Especially as it would provide a global view that spanned hundreds of projects. Whenever I received a threat alert I would very quickly be able to see root cause using SQ v5.1.2. Or causes. With a large enough set of projects, dependencies can have quite a lot of variety.

          Sure, the Dependencies Service UI was clunky... but it did work.

          Now that I have upgraded from v5.1.2 to v5.6 I am basically in the dark. It takes hours to manually trace dependencies via POMs. I could use an IDE but (speaking as a non-developer) that only seem to give analysis project by project. I would still not have the global perspective that is so desperately needed.

          Please, please bring back this functionality.

          Show
          marks Mark Symons added a comment - It's the Dependencies Service that I am particularly missing. I use Sonatype Nexus IQ to analyse our software for security (and other) threats. This is great at telling me that there is a threat from a particular component - but not good at telling me anything about dependencies. I also tried other products, such as Black Duck Hub and White Source... none were good at providing dependency information. This is where SonaQube Dependencies was so useful. Especially as it would provide a global view that spanned hundreds of projects. Whenever I received a threat alert I would very quickly be able to see root cause using SQ v5.1.2. Or causes. With a large enough set of projects, dependencies can have quite a lot of variety. Sure, the Dependencies Service UI was clunky... but it did work. Now that I have upgraded from v5.1.2 to v5.6 I am basically in the dark. It takes hours to manually trace dependencies via POMs. I could use an IDE but (speaking as a non-developer) that only seem to give analysis project by project. I would still not have the global perspective that is so desperately needed. Please, please bring back this functionality.
          Hide
          jrod John Rodriguez added a comment -

          > Few people are using and giving feedback on the Design features (some bugs were discovered very long after they were introduced)

          Technical debt cleanup is an often-abandoned part of software engineering, so it isn't surprising that feedback wasn't given. However, this was literally the feature that made me aware of Sonar years ago. I'm now revisiting a domain complexity problem and am disappointed to see that this is no longer a feature. The alternative is to tinker with degraph and yed (overly primitive tools) to analyze my project's DSM.

          What is the level of effort to upkeep? I'd be interested in OSS contributions if open for discussion.

          Show
          jrod John Rodriguez added a comment - > Few people are using and giving feedback on the Design features (some bugs were discovered very long after they were introduced) Technical debt cleanup is an often-abandoned part of software engineering, so it isn't surprising that feedback wasn't given. However, this was literally the feature that made me aware of Sonar years ago. I'm now revisiting a domain complexity problem and am disappointed to see that this is no longer a feature. The alternative is to tinker with degraph and yed (overly primitive tools) to analyze my project's DSM. What is the level of effort to upkeep? I'd be interested in OSS contributions if open for discussion.
          Show
          jrod John Rodriguez added a comment - Here's the commit for SONAR-6554 for those interested: https://github.com/SonarSource/sonarqube/commit/3c31c9522af4fe05040ac681f638a81d34a23387
          Show
          jrod John Rodriguez added a comment - Here's the commit for SONAR-6555 : https://github.com/SonarSource/sonarqube/commit/1abbd252c8513f92dbb9875288c5d000bb8f8c29
          Hide
          jrod John Rodriguez added a comment -
          Show
          jrod John Rodriguez added a comment - Here's the commit range for SONAR-6557 : https://github.com/SonarSource/sonarqube/commit/00f29b37b53df77bee37329354d46619ee2a09a7 https://github.com/SonarSource/sonarqube/commit/commit 81c539e66ba0b32109ca48fd61ab8dc54d8d95e5 https://github.com/SonarSource/sonarqube/commit/commit 3cab34083fc3e64a64982acc3ca28141d4da1f18 https://github.com/SonarSource/sonarqube/commit/commit c6233985ff4a9715566def58c8dcd2e7821755e6
          freddy.mallet Freddy Mallet made changes -
          Workflow Default Agile SonarSource Workflow V5 [ 183794 ] Default Agile SonarSource Workflow V6 [ 229951 ]
          freddy.mallet Freddy Mallet made changes -
          Workflow Default Agile SonarSource Workflow V6 [ 229951 ] Default Agile SonarSource Workflow V7 [ 259142 ]

            People

            • Assignee:
              julien.lancelot Julien Lancelot
              Reporter:
              fabemn OLD - Fabrice Bellingard
            • Votes:
              0 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

              • Due:
                Created:
                Updated:
                Resolved: