Execute decorators failing

[guwirth]

Have some projects which are failing after Execute decorators (SQ 5.1.1 & CXX 0.9.4). I don't know why? Someone an idea?

13:02:36.129 INFO  - Execute decorators...
13:02:36.129 DEBUG - Decorators: ManualMeasureDecorator -> QProfileEventsDecorator -> org.sonar.batch.design.ProjectDsmDecorator@7dc56c0a -> org.sonar.batch.design.SubProjectDsmDecorator@6c4de249 -> org.sonar.batch.design.DirectoryDsmDecorator@3c73a08e -> FileTangleIndexDecorator -> SumDuplicationsDecorator -> UnitTestDecorator -> org.sonar.plugins.core.security.ApplyProjectRolesDecorator@4f8365b2 -> org.sonar.plugins.core.sensors.DirectoriesDecorator@73990c4d -> org.sonar.plugins.core.sensors.FilesDecorator@5a2adcd -> org.sonar.plugins.core.timemachine.TimeMachineConfigurationPersister@27e05d64 -> org.sonar.plugins.core.timemachine.NewCoverageAggregator@6e3a9891 -> f(lines) -> f(generated_lines) -> f(ncloc) -> f(generated_ncloc) -> f(classes) -> f(packages) -> f(functions) -> f(accessors) -> f(statements) -> f(public_api) -> f(comment_lines) -> f(public_undocumented_api) -> f(commented_out_code_lines) -> f(complexity) -> f(complexity_in_classes) -> f(complexity_in_functions) -> f(class_complexity_distribution) -> f(function_complexity_distribution) -> f(file_complexity_distribution) -> f(lines_to_cover) -> f(uncovered_lines) -> f(conditions_to_cover) -> f(uncovered_conditions) -> f(it_lines_to_cover) -> f(it_uncovered_lines) -> f(it_conditions_to_cover) -> f(it_uncovered_conditions) -> f(overall_lines_to_cover) -> f(overall_uncovered_lines) -> f(overall_conditions_to_cover) -> f(overall_uncovered_conditions) -> f(rfc) -> f(rfc_distribution) -> f(lcom4_distribution) -> f(package_cycles) -> f(package_tangles) -> f(package_feedback_edges) -> f(package_edges_weight) -> f(new_lines_to_cover) -> f(new_uncovered_lines) -> f(new_conditions_to_cover) -> f(new_uncovered_conditions) -> f(new_it_lines_to_cover) -> f(new_it_uncovered_lines) -> f(new_it_conditions_to_cover) -> f(new_it_uncovered_conditions) -> f(new_overall_lines_to_cover) -> f(new_overall_uncovered_lines) -> f(new_overall_conditions_to_cover) -> f(new_overall_uncovered_conditions) -> org.sonar.batch.language.LanguageDistributionDecorator@1b19e5bf -> DirectoryTangleIndexDecorator -> org.sonar.batch.cpd.decorators.DuplicationDensityDecorator@6f7cc804 -> CommentDensityDecorator -> f(file_complexity) -> f(class_complexity) -> f(function_complexity) -> org.sonar.plugins.core.sensors.LineCoverageDecorator@fdc872e -> org.sonar.plugins.core.sensors.CoverageDecorator@b484c37 -> org.sonar.plugins.core.sensors.BranchCoverageDecorator@71f1ea35 -> org.sonar.plugins.core.sensors.ItLineCoverageDecorator@3d4cae3f -> org.sonar.plugins.core.sensors.ItCoverageDecorator@5642ab4f -> org.sonar.plugins.core.sensors.ItBranchCoverageDecorator@1f227c44 -> org.sonar.plugins.core.sensors.OverallLineCoverageDecorator@617c9dd7 -> org.sonar.plugins.core.sensors.OverallCoverageDecorator@3999ea91 -> org.sonar.plugins.core.sensors.OverallBranchCoverageDecorator@7ac2263e -> Common Rules for c++ -> org.sonar.batch.issue.tracking.IssueTrackingDecorator@7aa5596f -> org.sonar.batch.debt.DebtDecorator@4778db1f -> org.sonar.batch.debt.NewDebtDecorator@3d356e85 -> CountUnresolvedIssuesDecorator -> CountFalsePositivesDecorator -> org.sonar.batch.debt.SqaleRatingDecorator@3e159f12 -> com.gistlabs.sonar.plugins.dplplugin.DPLDecorator@6d9c656d -> org.sonar.plugins.core.timemachine.TendencyDecorator@6d093764 -> VariationDecorator
13:02:36.178 DEBUG - Release semaphore on project : org.sonar.api.resources.Project@357a3cb9[id=1533,key=XXX,qualifier=TRK], with key batch-XXX
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
Total time: 12.230s
Final Memory: 23M/823M
INFO: ------------------------------------------------------------------------

Normally it should continue like this:

13:02:46.422 DEBUG - Decorator time:
    ManualMeasureDecorator: 188ms
    QProfileEventsDecorator: 10ms
    org.sonar.batch.design.ProjectDsmDecorator@664a3770: 0ms
    org.sonar.batch.design.SubProjectDsmDecorator@63a7a23a: 1ms
    org.sonar.batch.design.DirectoryDsmDecorator@785e236f: 2ms
    FileTangleIndexDecorator: 0ms
    SumDuplicationsDecorator: 2ms
    UnitTestDecorator: 0ms
    org.sonar.plugins.core.security.ApplyProjectRolesDecorator@48afc94c: 3ms
    org.sonar.plugins.core.sensors.DirectoriesDecorator@51dc7140: 0ms
    org.sonar.plugins.core.sensors.FilesDecorator@d9347e: 1ms
    org.sonar.plugins.core.timemachine.TimeMachineConfigurationPersister@1a0108a5: 4ms
    org.sonar.plugins.core.timemachine.NewCoverageAggregator@7e88ba07: 1ms
    f(lines): 0ms
    f(generated_lines): 0ms
    f(ncloc): 0ms
    f(generated_ncloc): 0ms
    f(classes): 0ms
    f(packages): 0ms
    f(functions): 1ms
    f(accessors): 0ms
    f(statements): 0ms
    f(public_api): 0ms
    f(comment_lines): 0ms
    f(public_undocumented_api): 0ms
    f(commented_out_code_lines): 0ms
    f(complexity): 0ms
    f(complexity_in_classes): 0ms
    f(complexity_in_functions): 0ms
    f(class_complexity_distribution): 0ms
    f(function_complexity_distribution): 6ms
    f(file_complexity_distribution): 1ms
    f(lines_to_cover): 0ms
    f(uncovered_lines): 0ms
    f(conditions_to_cover): 0ms
    f(uncovered_conditions): 0ms
    f(it_lines_to_cover): 0ms
    f(it_uncovered_lines): 0ms
    f(it_conditions_to_cover): 0ms
    f(it_uncovered_conditions): 0ms
    f(overall_lines_to_cover): 0ms
    f(overall_uncovered_lines): 0ms
    f(overall_conditions_to_cover): 0ms
    f(overall_uncovered_conditions): 0ms
    f(rfc): 0ms
    f(rfc_distribution): 0ms
    f(lcom4_distribution): 0ms
    f(package_cycles): 0ms
    f(package_tangles): 0ms
    f(package_feedback_edges): 0ms
    f(package_edges_weight): 0ms
    f(new_lines_to_cover): 0ms
    f(new_uncovered_lines): 0ms
    f(new_conditions_to_cover): 1ms
    f(new_uncovered_conditions): 0ms
    f(new_it_lines_to_cover): 1ms
    f(new_it_uncovered_lines): 0ms
    f(new_it_conditions_to_cover): 0ms
    f(new_it_uncovered_conditions): 0ms
    f(new_overall_lines_to_cover): 0ms
    f(new_overall_uncovered_lines): 0ms
    f(new_overall_conditions_to_cover): 0ms
    f(new_overall_uncovered_conditions): 0ms
    org.sonar.batch.language.LanguageDistributionDecorator@4e023095: 6ms
    DirectoryTangleIndexDecorator: 0ms
    org.sonar.batch.cpd.decorators.DuplicationDensityDecorator@6afe49f5: 0ms
    CommentDensityDecorator: 1ms
    f(file_complexity): 1ms
    f(class_complexity): 0ms
    f(function_complexity): 0ms
    org.sonar.plugins.core.sensors.LineCoverageDecorator@1ff84804: 1ms
    org.sonar.plugins.core.sensors.CoverageDecorator@1fba5aad: 1ms
    org.sonar.plugins.core.sensors.BranchCoverageDecorator@4837635d: 1ms
    org.sonar.plugins.core.sensors.ItLineCoverageDecorator@472ea479: 0ms
    org.sonar.plugins.core.sensors.ItCoverageDecorator@1acb7cf0: 1ms
    org.sonar.plugins.core.sensors.ItBranchCoverageDecorator@57e92236: 1ms
    org.sonar.plugins.core.sensors.OverallLineCoverageDecorator@24378e1d: 1ms
    org.sonar.plugins.core.sensors.OverallCoverageDecorator@1608fb8e: 0ms
    org.sonar.plugins.core.sensors.OverallBranchCoverageDecorator@57653943: 0ms
    Common Rules for c++: 8ms
    org.sonar.batch.issue.tracking.IssueTrackingDecorator@7f9c9fce: 50ms
    org.sonar.batch.debt.DebtDecorator@5fc4773e: 378ms
    org.sonar.batch.debt.NewDebtDecorator@5a14cfa4: 3ms
    CountUnresolvedIssuesDecorator: 15ms
    CountFalsePositivesDecorator: 9ms
    org.sonar.batch.debt.SqaleRatingDecorator@1f76b33a: 4ms
    com.gistlabs.sonar.plugins.dplplugin.DPLDecorator@2d907fcc: 1ms
    org.sonar.plugins.core.timemachine.TendencyDecorator@1f0b18ed: 51ms
    VariationDecorator: 0ms

13:02:46.422 INFO  - Store results in database
13:02:46.422 DEBUG - Execute org.sonar.batch.phases.GraphPersister
13:02:46.430 DEBUG - Execute org.sonar.batch.index.SourcePersister
13:02:46.820 DEBUG - Execute org.sonar.batch.index.ResourcePersister
13:02:46.820 DEBUG - Execute org.sonar.batch.index.MeasurePersister
13:02:50.897 DEBUG - Updating semaphore batch-XXX
13:02:50.951 DEBUG - Execute org.sonar.batch.index.DuplicationPersister
13:02:51.043 INFO  - Analysis reports generated in 80ms, dir size=77 KB
13:02:51.105 INFO  - Analysis reports compressed in 62ms, zip size=48 KB
13:02:51.105 DEBUG - Publish results
13:02:51.712 INFO  - Analysis reports sent to server in 607ms
13:02:51.713 INFO  - ANALYSIS SUCCESSFUL, you can browse http://xxx.com:9000/dashboard/index/XXX
13:02:51.713 INFO  - Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report.
13:02:51.713 DEBUG - Release semaphore on project : org.sonar.api.resources.Project@44e49229[id=7320,key=XXX,qualifier=TRK], with key batch-XXX
...

[jmecosta]

usually when getting those in decorates there is more information about it. like that i havent seen

[guwirth]

Turning also error messages on (-e) I get info below. It is also an ArrayIndexOutOfBoundsException and the same as #600 (https://jira.sonarsource.com/browse/SONAR-6792).

I documented how to create additional info in https://github.com/wenns/sonar-cxx/wiki/Get-debug-information.

ERROR: Error during Sonar runner execution
org.sonar.runner.impl.RunnerException: Unable to execute Sonar
    at org.sonar.runner.impl.BatchLauncher$1.delegateExecution(BatchLauncher.java:91)
    at org.sonar.runner.impl.BatchLauncher$1.run(BatchLauncher.java:75)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.sonar.runner.impl.BatchLauncher.doExecute(BatchLauncher.java:69)
    at org.sonar.runner.impl.BatchLauncher.execute(BatchLauncher.java:50)
    at org.sonar.runner.api.EmbeddedRunner.doExecute(EmbeddedRunner.java:102)
    at org.sonar.runner.api.Runner.execute(Runner.java:100)
    at org.sonar.runner.Main.executeTask(Main.java:70)
    at org.sonar.runner.Main.execute(Main.java:59)
    at org.sonar.runner.Main.main(Main.java:53)
Caused by: org.sonar.api.utils.SonarException: Fail to decorate 'org.sonar.api.resources.File@28d1d634[key=HardwareAccess/HardwareAccess/AddOns/Mediators/DeviceConfigX/EtasDeviceConfigX.Com/HashedConfigList.h,path=HardwareAccess/HardwareAccess/AddOns/Mediators/DeviceConfigX/EtasDeviceConfigX.Com/HashedConfigList.h,filename=HashedConfigList.h,language=c++]'
    at org.sonar.batch.phases.DecoratorsExecutor.executeDecorator(DecoratorsExecutor.java:104)
    at org.sonar.batch.phases.DecoratorsExecutor.decorateResource(DecoratorsExecutor.java:87)
    at org.sonar.batch.phases.DecoratorsExecutor.decorateResource(DecoratorsExecutor.java:79)
    at org.sonar.batch.phases.DecoratorsExecutor.decorateResource(DecoratorsExecutor.java:79)
    at org.sonar.batch.phases.DecoratorsExecutor.execute(DecoratorsExecutor.java:71)
    at org.sonar.batch.phases.DatabaseModePhaseExecutor.execute(DatabaseModePhaseExecutor.java:127)
    at org.sonar.batch.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:264)
    at org.sonar.api.platform.ComponentContainer.startComponents(ComponentContainer.java:92)
    at org.sonar.api.platform.ComponentContainer.execute(ComponentContainer.java:77)
    at org.sonar.batch.scan.ProjectScanContainer.scan(ProjectScanContainer.java:235)
    at org.sonar.batch.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:230)
    at org.sonar.batch.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:220)
    at org.sonar.api.platform.ComponentContainer.startComponents(ComponentContainer.java:92)
    at org.sonar.api.platform.ComponentContainer.execute(ComponentContainer.java:77)
    at org.sonar.batch.scan.ScanTask.scan(ScanTask.java:57)
    at org.sonar.batch.scan.ScanTask.execute(ScanTask.java:45)
    at org.sonar.batch.bootstrap.TaskContainer.doAfterStart(TaskContainer.java:135)
    at org.sonar.api.platform.ComponentContainer.startComponents(ComponentContainer.java:92)
    at org.sonar.api.platform.ComponentContainer.execute(ComponentContainer.java:77)
    at org.sonar.batch.bootstrap.GlobalContainer.executeTask(GlobalContainer.java:158)
    at org.sonar.batch.bootstrapper.Batch.executeTask(Batch.java:95)
    at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:67)
    at org.sonar.runner.batch.IsolatedLauncher.execute(IsolatedLauncher.java:48)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.sonar.runner.impl.BatchLauncher$1.delegateExecution(BatchLauncher.java:87)
    ... 9 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 80
    at org.sonar.batch.issue.tracking.FileHashes.getHash(FileHashes.java:89)
    at org.sonar.batch.issue.tracking.IssueTracking.setChecksumOnNewIssues(IssueTracking.java:66)
    at org.sonar.batch.issue.tracking.IssueTracking.track(IssueTracking.java:51)
    at org.sonar.batch.issue.tracking.IssueTrackingDecorator.doDecorate(IssueTrackingDecorator.java:139)
    at org.sonar.batch.issue.tracking.IssueTrackingDecorator.decorate(IssueTrackingDecorator.java:113)
    at org.sonar.batch.phases.DecoratorsExecutor.executeDecorator(DecoratorsExecutor.java:96)
    ... 36 more

[jmecosta]

#611 it should be related with this. we also get it once in a while when sources and reports get unsynched

[alinddicu]

Hello!

We're also having this problem with sonar-runner-2.4 and sonarqube-5.1.2 . It seems indeed like sources and reports are unsynchronized.

I tried to workaround by adding empty lines at the end of the source files and have the runner complete, but there could be a lot of source files in this case.

Is there another way to solve the cause of the problem: unsynchronized sources and reports?

[jmecosta]

We get this because you don't run static analysis in the same moment as we run sonar analysis because of the nature of our projects. But we have this well identified and actually the failing builds actually are a good thing since it prevent adding incorrect data

[alinddicu]

Actually there were only 3 files concerned, but one can't know that before analysis. There could have been a lot more... The runner succeeded and the reports were saved to the database. I removed the extra lines and scheduled another run. Wait and see :)

So you mean that the unsync is an identified problem with work in progress?

[jmecosta]

yep, but its our own context. should not apply to you

[alinddicu]

Ok, thanks anyway!

This workaround was very helpful!