README 2.03 KB
Newer Older
Philippe Pepiot's avatar
Philippe Pepiot committed
1
2
Summary
-------
Philippe Pepiot's avatar
Philippe Pepiot committed
3
4
5
6

Run, monitor and log celery tasks.


7
8
Installation and setup
----------------------
Philippe Pepiot's avatar
Philippe Pepiot committed
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

Declare tasks using celery task or cubicweb-celery cwtasks.

On worker side, install cw-celerytask-helpers from ./helpers.

celeryconfig.py example::

    BROKER_URL = 'redis://localhost:6379/0'
    CELERY_RESULT_BACKEND = BROKER_URL
    CUBICWEB_CELERYTASK_REDIS_URL = BROKER_URL
    CELERY_IMPORTS = ('cw_celerytask_helpers', 'module.containing.tasks')
    CELERY_TASK_SERIALIZER = 'json'
    CELERY_RESULT_SERIALIZER = 'json'
    CELERY_ACCEPT_CONTENT = ['json', 'msgpack', 'yaml']


Start a worker::

    # running cubicweb tasks (celeryconfig.py will be imported from your instance config directory)
    celery -A cubicweb_celery -i <CW_INSTANCE_NAME> worker -l info -E

    # running pure celery tasks
    celery worker -l info -E


34
35
36
37
38
Task state synchronization requires to run the `celery-monitor` command::

    cubicweb-ctl celery-monitor <instance-name>


39
40
41
42
43
Ensure to have the celeryconfig.py loaded for both cubicweb instance and
celery worker, enforce by settings with CELERY_CONFIG_MODULE environment
variable (it must be an importable python module).


44
45
46
47
Running tasks
-------------


Philippe Pepiot's avatar
Philippe Pepiot committed
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
Create a task:

.. code-block:: python

    from celery import current_app as app
    from celery.utils.log import get_task_logger

    logger = get_task_logger(__name__)

    @app.task(name='hi_there')
    def my_task(arg, kw=0):
        logger.info('HI %s %s!', arg, kw)
        return 42


Run a task:

.. code-block:: python

    from cubes.celerytask.entities import start_async_task

    cwtask = start_async_task(cnx, 'hi_there', 'THERE', kw=42)
70
    cnx.commit()
71
72
73
74
75


start_async_task() accept task names, task objects or task signatures:
http://docs.celeryproject.org/en/latest/userguide/canvas.html#signatures

76
77
78
For instance, to start the above task in a dedicated queue named `myqueue`:

.. code-block:: python
79

80
    import celery
81

82
83
    start_async_task(cnx, celery.signature('hi_there', args=('THERE',),
                                           kwargs={'kw': 42}, queue='myqueue'))