-
Type:
MMF
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Labels:
Context
Currently, a plugin can request the indexing of files with certain extensions, e.g. .java, and the platform will index those files and hand them off to the requester. However only a single plugin may register a given extension, and files with extensions that have not been requested by any plugin are simply unavailable (unless the indexing of all files has been enabled). As a consequence, plugins cannot create issues on random files.
Purpose of the MMF
A language plugin should be able to analyse any file that exists in the project - whatever the extension of that file, and create issues on those files.
Concrete example:
- The Java plugin wants to be able to create issues on the pom.xml or web.xml files if they exist in the project
- The Java plugin should be able to work on *.xml files without causing a conflict with the XML plugin
- The PHP plugin wants to be able to create issues on the config.ini file if it exists
Proposal
- Index by default all files that are part of source sets, not just the ones matching file suffixes declared by Language extension points (same as using sonar.import_unknown_files=true)
- continue to honor sonar.sources/sonar.tests/sonar.exclusions/... -> update scanners if needed to have more files in the source sets (like src/main/* in Maven)
- To avoid reading content of all indexed files (performance penalty + possible failure on binary files):
- delay initialization of file metadata (lines / charset / offset per line). The first sensor trying to access one of those attribute will trigger initialization. Have a "secret" property to revert to systematic metadata initialization to not break language team performance ITs.
- only blame / compute CPD on files to be published (see next item)
- optimize some FS query (and introduce dedicated predicates). It should be cheap to query by file extension or by filename.
- To not bloat scanner report with too many files, only publish:
- "touched" files = files for which a Sensor had stored some data (measure/issue/highlighting/...)
- files could be explicitly marked for inclusion in the report using a new API (e.g. sensorContext.markForPublishing(inputFile)): this is to avoid a file without language to disappear when all issues have been fixed
Example for the JAVA and PHP plugins
- On 6.3 with non-upgraded plugin: FS query will return an InputFile (assuming it is part of scanner source sets). If at least one issue is saved, the file will be published.
- On 6.3 with upgraded plugin: Java plugin can force publishing of all **/pom.xml using new API to have better consistency (number of files in the project will not change depending on if an issue is raised or not)
for (InputFile f : fs.inputFiles(p.matchesPathPattern("**/*.xml")) { if (f.path().endsWith("pom.xml") { if (context.getSonarQubeVersion().isGreaterThanOrEquals("6.3")) { context.markForPublishing(f); } executePomChecks(f); } else if (f.path().endsWith("ejb-jar.xml")) { if (context.getSonarQubeVersion().isGreaterThanOrEquals("6.3")) { context.markForPublishing(f); } executeEJBChecks(f); } ... }
- breaks down into
-
SONAR-8622 Lazily generate metadata for input files
-
- Closed
-
-
SONAR-8623 Index all source files, even the ones with no supported language
-
- Closed
-
-
SONAR-8630 Optimize queries of InputFiles by extensions or filename
-
- Closed
-
-
SONAR-8631 Only publish input files used by sensors
-
- Closed
-
- is related to
-
MMF-672 Better support of multi-language per file extension
-
- In Specification
-