README 1.7 KB
Newer Older
Philippe Pepiot's avatar
Philippe Pepiot committed
1
2
Summary
-------
Philippe Pepiot's avatar
Philippe Pepiot committed
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

Run, monitor and log celery tasks.


Usage
-----

Declare tasks using celery task or cubicweb-celery cwtasks.

On worker side, install cw-celerytask-helpers from ./helpers.

celeryconfig.py example::

    BROKER_URL = 'redis://localhost:6379/0'
    CELERY_RESULT_BACKEND = BROKER_URL
    CUBICWEB_CELERYTASK_REDIS_URL = BROKER_URL
    CELERY_IMPORTS = ('cw_celerytask_helpers', 'module.containing.tasks')
    CELERY_TASK_SERIALIZER = 'json'
    CELERY_RESULT_SERIALIZER = 'json'
    CELERY_ACCEPT_CONTENT = ['json', 'msgpack', 'yaml']


Start a worker::

    # running cubicweb tasks (celeryconfig.py will be imported from your instance config directory)
    celery -A cubicweb_celery -i <CW_INSTANCE_NAME> worker -l info -E

    # running pure celery tasks
    celery worker -l info -E


34
35
36
37
38
Ensure to have the celeryconfig.py loaded for both cubicweb instance and
celery worker, enforce by settings with CELERY_CONFIG_MODULE environment
variable (it must be an importable python module).


Philippe Pepiot's avatar
Philippe Pepiot committed
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Create a task:

.. code-block:: python

    from celery import current_app as app
    from celery.utils.log import get_task_logger

    logger = get_task_logger(__name__)

    @app.task(name='hi_there')
    def my_task(arg, kw=0):
        logger.info('HI %s %s!', arg, kw)
        return 42


Run a task:

.. code-block:: python

    from cubes.celerytask.entities import start_async_task

    cwtask = start_async_task(cnx, 'hi_there', 'THERE', kw=42)
61
    cnx.commit()
62
63
64
65
66
67
68
69


start_async_task() accept task names, task objects or task signatures:
http://docs.celeryproject.org/en/latest/userguide/canvas.html#signatures


Task state synchronization require to run the `celery-monitor` command::

70
    cubicweb-ctl celery-monitor <instance-name>