GitLab

We should not stop celery-monitor in case of temporary unavailability of redis
or cubicweb database.
Handle errors and automatically retry each 5 seconds.

Previously we handle monitoring celery task by listening to celery event bus
(celery.events.EventReceiver) that was not persistent. In this case we used a
dedicated routine (on_monitor_start) to synchronize non finished tasks, but
this wasn't working in case of an untracked task (eg. not started with
start_async_task). Also this was a single point of failure because it cannot
run in multiple instance without concurrency issues (events are sent to all
instances)

Now we use a redis queue where worker put task_id and task_name to be
synchronized and celery-monitor use brpop (https://redis.io/commands/brpop) to
process the queue.

We don't require CELERY_SEND_EVENTS to be enabled anymore (-E or --events in
worker options).
We require to add 'cw_celerytask_helpers.helpers' to CELERY_IMPORTS.

Do not create a CeleryTask if no worker has aknowledge the said task.

it may occur that a transition cannot be fired due to a hook triggered by a
previous transition. Especially the on_monitor_start() was prone to this kind
of error since it was doing all the transitions in a single transaction.

Adapters based API
Inspired from https://hg.logilab.org/users/david/cw-celery-logging-demo/