This mechanism should be a standard way to:
- take the next item to consume from the queue
- run some "actions" on this item
- somehow log the result of its execution:
- If everything went fine, just an OK status with a timestamp
- When there are failures, these should be stored somehow (so that it's possible to see them in the admin console)
This mechanism should not impact the Web server:
- if there's a failure during an action or even an unexpected error in the mechanism itself, this should not shut down the whole web server
- this mechanism should be able to auto-recover from unexpected errors, i.e. the user should not have to worry about this service being up or not
At any moment of time, it should be possible to know:
- the status of this mechanism (idle, working)
- the current item being processed
The validation of this step will be to move the current actions of the "/batch/upload_report" WS (sync of issues in E/S) to this new worker mechanism. At this point of time, the "/batch/upload_report" WS should therefore only push an item to the queue and then return in order to not block the batch.
Here too, we should put some information in the log this so that it's easy to verify that this is working (until we have the feedback in the admin console).
Don't forget to add a cleanup step at the server startup for the job which were working