Uploaded image for project: 'SonarQube'
  1. SonarQube
  2. SONAR-10943

Analysis may be kept pending indefinitely in Compute Engine on PostgreSQL

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 6.7.1
    • Fix Version/s: 6.7.5, 7.3
    • Component/s: Compute Engine
    • Labels:
    • Edition:
      Community
    • Production Notes:
      None

      Description

      Reproducer

      1. connect SonarQube < 7.2 to a PostgreSQL database
      2. install the attached Xoo plugin
      3. activate the rule Issue With Custom Message"
      4. analyze a xoo project (example) in order to generate an issue with a message than contains the null character `\u0000`. This character can't be persisted on PostgreSQL and makes the compute engine task fail (see SONAR-10460).
      5. the task is logged as finished in logs/ce.log but it remains "in-progress" in Administration > Projects > Background Tasks.
        2018.06.26 16:02:17 INFO  ce[AWQ8Zu0dgHlsBm80KawY][o.s.c.t.CeWorkerImpl] Execute task | project=sample | type=REPORT | id=AWQ8Zu0dgHlsBm80KawY
        2018.06.26 16:02:17 ERROR ce[AWQ8Zu0dgHlsBm80KawY][o.s.c.t.CeWorkerImpl] Failed to execute task AWQ8Zu0dgHlsBm80KawY
        org.apache.ibatis.exceptions.PersistenceException: 
        ### Error flushing statements.  Cause: org.apache.ibatis.executor.BatchExecutorException: org.sonar.db.issue.IssueMapper.insert (batch index #1) failed. Cause: java.sql.BatchUpdateException: Batch entry 0 INSERT INTO issues (kee, rule_id, severity, manual_severity,
            message, line, locations, gap, effort, status, tags,
            resolution, checksum, assignee, author_login, issue_attributes, issue_creation_date, issue_update_date,
            issue_close_date, created_at, updated_at, component_uuid, project_uuid, issue_type)
            VALUES ('AWQ8ZvRj78iACyoXc27p', 968,
            'MAJOR',
            'FALSE', 'invalid invalid  message', NULL,
            ?,
            NULL, NULL, 'OPEN',
            NULL, NULL, '',
            NULL, 'julien.lancelot@sonarsource.com',
            '',
            1530021734000,1530021734000, NULL,
            1530021737654, 1530021737654,
            'AWQ8ZfeZ78iACyoXc27j', 'AWQ8ZfKJgHlsBm80KawR', 1) was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00  Call getNextException to see other errors in the batch.
        ### Cause: org.apache.ibatis.executor.BatchExecutorException: org.sonar.db.issue.IssueMapper.insert (batch index #1) failed. Cause: java.sql.BatchUpdateException: Batch entry 0 INSERT INTO issues (kee, rule_id, severity, manual_severity,
            message, line, locations, gap, effort, status, tags,
            resolution, checksum, assignee, author_login, issue_attributes, issue_creation_date, issue_update_date,
            issue_close_date, created_at, updated_at, component_uuid, project_uuid, issue_type)
            VALUES ('AWQ8ZvRj78iACyoXc27p', 968,
            'MAJOR',
            'FALSE', 'invalid invalid  message', NULL,
            ?,
            NULL, NULL, 'OPEN',
            NULL, NULL, '',
            NULL, 'julien.lancelot@sonarsource.com',
            '',
            1530021734000,1530021734000, NULL,
            1530021737654, 1530021737654,
            'AWQ8ZfeZ78iACyoXc27j', 'AWQ8ZfKJgHlsBm80KawR', 1) was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00  Call getNextException to see other errors in the batch.
        	at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)
        	at org.apache.ibatis.session.defaults.DefaultSqlSession.flushStatements(DefaultSqlSession.java:255)
        	at org.sonar.db.DbSessionImpl.flushStatements(DbSessionImpl.java:167)
        	at org.sonar.db.BatchSession.flushStatements(BatchSession.java:192)
        	at org.sonar.server.computation.task.projectanalysis.step.PersistIssuesStep.execute(PersistIssuesStep.java:67)
        	at org.sonar.server.computation.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:64)
        	at org.sonar.server.computation.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:52)
        	at org.sonar.server.computation.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:73)
        	at org.sonar.ce.taskprocessor.CeWorkerImpl.executeTask(CeWorkerImpl.java:134)
        	at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:97)
        	at org.sonar.ce.taskprocessor.CeWorkerImpl.withCustomizedThreadName(CeWorkerImpl.java:81)
        	at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:73)
        	at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:43)
        	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        	at java.lang.Thread.run(Thread.java:748)
        Caused by: org.apache.ibatis.executor.BatchExecutorException: org.sonar.db.issue.IssueMapper.insert (batch index #1) failed. Cause: java.sql.BatchUpdateException: Batch entry 0 INSERT INTO issues (kee, rule_id, severity, manual_severity,
            message, line, locations, gap, effort, status, tags,
            resolution, checksum, assignee, author_login, issue_attributes, issue_creation_date, issue_update_date,
            issue_close_date, created_at, updated_at, component_uuid, project_uuid, issue_type)
            VALUES ('AWQ8ZvRj78iACyoXc27p', 968,
            'MAJOR',
            'FALSE', 'invalid invalid  message', NULL,
            ?,
            NULL, NULL, 'OPEN',
            NULL, NULL, '',
            NULL, 'julien.lancelot@sonarsource.com',
            '',
            1530021734000,1530021734000, NULL,
            1530021737654, 1530021737654,
            'AWQ8ZfeZ78iACyoXc27j', 'AWQ8ZfKJgHlsBm80KawR', 1) was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00  Call getNextException to see other errors in the batch.
        	at org.apache.ibatis.executor.BatchExecutor.doFlushStatements(BatchExecutor.java:146)
        	at org.apache.ibatis.executor.BaseExecutor.flushStatements(BaseExecutor.java:129)
        	at org.apache.ibatis.executor.BaseExecutor.flushStatements(BaseExecutor.java:122)
        	at org.apache.ibatis.executor.CachingExecutor.flushStatements(CachingExecutor.java:114)
        	at org.apache.ibatis.session.defaults.DefaultSqlSession.flushStatements(DefaultSqlSession.java:253)
        	... 19 common frames omitted
        Caused by: java.sql.BatchUpdateException: Batch entry 0 INSERT INTO issues (kee, rule_id, severity, manual_severity,
            message, line, locations, gap, effort, status, tags,
            resolution, checksum, assignee, author_login, issue_attributes, issue_creation_date, issue_update_date,
            issue_close_date, created_at, updated_at, component_uuid, project_uuid, issue_type)
            VALUES ('AWQ8ZvRj78iACyoXc27p', 968,
            'MAJOR',
            'FALSE', 'invalid invalid  message', NULL,
            ?,
            NULL, NULL, 'OPEN',
            NULL, NULL, '',
            NULL, 'julien.lancelot@sonarsource.com',
            '',
            1530021734000,1530021734000, NULL,
            1530021737654, 1530021737654,
            'AWQ8ZfeZ78iACyoXc27j', 'AWQ8ZfKJgHlsBm80KawR', 1) was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00  Call getNextException to see other errors in the batch.
        	at org.postgresql.jdbc.BatchResultHandler.handleError(BatchResultHandler.java:148)
        	at org.postgresql.core.ResultHandlerDelegate.handleError(ResultHandlerDelegate.java:50)
        	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2168)
        	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:479)
        	at org.postgresql.jdbc.PgStatement.executeBatch(PgStatement.java:835)
        	at org.postgresql.jdbc.PgPreparedStatement.executeBatch(PgPreparedStatement.java:1556)
        	at org.apache.commons.dbcp.DelegatingStatement.executeBatch(DelegatingStatement.java:297)
        	at org.apache.commons.dbcp.DelegatingStatement.executeBatch(DelegatingStatement.java:297)
        	at org.apache.ibatis.executor.BatchExecutor.doFlushStatements(BatchExecutor.java:122)
        	... 23 common frames omitted
        Caused by: org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00
        	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2422)
        	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2167)
        	... 29 common frames omitted
        2018.06.26 16:02:17 ERROR ce[AWQ8Zu0dgHlsBm80KawY][o.s.c.t.CeWorkerImpl] Failed to finalize task with uuid 'AWQ8Zu0dgHlsBm80KawY' and persist its state to db
        org.apache.ibatis.exceptions.PersistenceException: 
        ### Error updating database.  Cause: org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00
        ### The error may involve org.sonar.db.ce.CeActivityMapper.insert-Inline
        ### The error occurred while setting parameters
        ### SQL: insert into ce_activity (       uuid,       component_uuid,       analysis_uuid,       status,       task_type,       is_last,       is_last_key,       submitter_login,       submitted_at,       worker_uuid,       execution_count,       started_at,       executed_at,       created_at,       updated_at,       execution_time_ms,       error_message,       error_stacktrace,       error_type     )     values (       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?     )
        ### Cause: org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00
        	at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)
        	at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:200)
        	at org.apache.ibatis.session.defaults.DefaultSqlSession.insert(DefaultSqlSession.java:185)
        	at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:57)
        	at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:59)
        	at com.sun.proxy.$Proxy27.insert(Unknown Source)
        	at org.sonar.db.ce.CeActivityDao.insert(CeActivityDao.java:55)
        	at org.sonar.ce.queue.CeQueueImpl.remove(CeQueueImpl.java:156)
        	at org.sonar.ce.queue.InternalCeQueueImpl.remove(InternalCeQueueImpl.java:110)
        	at org.sonar.ce.taskprocessor.CeWorkerImpl.finalizeTask(CeWorkerImpl.java:165)
        	at org.sonar.ce.taskprocessor.CeWorkerImpl.executeTask(CeWorkerImpl.java:148)
        	at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:97)
        	at org.sonar.ce.taskprocessor.CeWorkerImpl.withCustomizedThreadName(CeWorkerImpl.java:81)
        	at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:73)
        	at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:43)
        	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        	at java.lang.Thread.run(Thread.java:748)
        Caused by: org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00
        	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2422)
        	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2167)
        	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306)
        	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
        	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
        	at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:155)
        	at org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:144)
        	at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
        	at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
        	at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:46)
        	at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:74)
        	at org.apache.ibatis.executor.ReuseExecutor.doUpdate(ReuseExecutor.java:52)
        	at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117)
        	at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:76)
        	at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:198)
        	... 21 common frames omitted
        2018.06.26 16:02:17 ERROR ce[AWQ8Zu0dgHlsBm80KawY][o.s.c.t.CeWorkerImpl] Executed task | project=sample | type=REPORT | id=AWQ8Zu0dgHlsBm80KawY | time=307ms
        
      6. analyze a non-xoo project. The previous xoo task is reverted to pending status, and is kept in this status indefinitely, even on server restart.

      Example of db row in inconsistent state :

      INSERT INTO ce_queue (uuid,task_type,component_uuid,status,submitter_login,started_at,created_at,updated_at,worker_uuid,execution_count) VALUES 
      ('AWQ8Zu0dgHlsBm80KawY','REPORT','AWQ8ZfKJgHlsBm80KawR','IN_PROGRESS',NULL,1530021737365,1530021735713,1530021737365,'AWQ8ZOpz78iACyoXc228',1)
      ;
      

      The root cause is that Compute Engine fails to persist the task error message but it contains the \u0000 null character too. The task is in an inconsistent state in database which prevents it to be processed.

      This bug impacts only PostgreSQL installations. The workaround is to restart the server.

        Attachments

          Activity

            People

            • Assignee:
              sebastien.lesaint Sebastien Lesaint
              Reporter:
              simon.brandhof Simon Brandhof
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Due:
                Created:
                Updated:
                Resolved: