1. 30 Jan, 2019 1 commit
  2. 29 Jan, 2019 8 commits
  3. 23 Aug, 2018 3 commits
  4. 26 Jun, 2018 3 commits
  5. 22 Jun, 2018 1 commit
    • Philippe Pepiot's avatar
      Migrate task logs from redis and database to logs files · e501a370ed29
      Philippe Pepiot authored
      Having logs stored in redis then in database took too much memory in redis and
      storage in database.
      Using files is far simplier, but it require to have a shared file system (nfs)
      when the worker and the cubiweb instance (the reader) are not in the same
      server.
      
      Use the new cw_celerytask_helpers filelogger instead of redislogger.
      Logs are stored in celerytask-log-dir directory in gzip with a predictible
      filename based on task_id (which is unique).
      
      Drop task_logs attribute from CeleryTask and update tests accordingly.
      celery-monitor don't copy anymore from redis to database when the task is
      ended.
      e501a370ed29
  6. 20 Jun, 2018 1 commit
    • Philippe Pepiot's avatar
      Add a new celerytask-log-dir option · 31147960ad58
      Philippe Pepiot authored
      We're going to introduce a new file based logger for celery task logs.
      Add a new 'celerytask-log-dir' option that is used to store those logs files.
      
      The CUBICWEB_CELERYTASK_LOGDIR option has been introduced in
      cw-celerytask-helpers for non-cubicweb workers (i.e. celery workers that
      doesn't require a cubicweb connexion).
      
      We must ensure that both options are consistent and set the the appropriate
      value, so a startup hook ensure the configuration is correct.
      31147960ad58
  7. 26 Jun, 2018 2 commits
  8. 21 Jun, 2018 1 commit
  9. 20 Jun, 2018 1 commit
  10. 06 Mar, 2018 2 commits
  11. 28 Feb, 2018 3 commits
  12. 22 Feb, 2018 1 commit
    • Philippe Pepiot's avatar
      Revoke and remove logs for deleted tasks · 0a4fd2cc714a
      Philippe Pepiot authored
      When a CeleryTask is deleted a postcommit hooks is trigerred to revoke the
      tasks (and child tasks) and to remove the logs from the redis database.
      
      This avoid filling the redis database with deleted task logs.
      0a4fd2cc714a
  13. 17 Nov, 2017 2 commits
  14. 15 Nov, 2017 2 commits
    • Philippe Pepiot's avatar
      Fix testing tasks creating other tasks · 1ef783692e79
      Philippe Pepiot authored
      When a task create a new task (by calling start_async_task), _TEST_TASK was
      reset during the loop on it, this was leading to a KeyError (in case of
      multiple tasks) or in sub-tasks not being started.
      
      Fix this by not overriding _TEST_TASK for each new cubicweb connection and by
      consuming _TEST_TASK until there is no tasks left.
      1ef783692e79
    • Philippe Pepiot's avatar
      Drop useless setUp() in BaseCeleryTaskTC · 0aca21e47a99
      Philippe Pepiot authored
      CELERY_ALWAYS_EAGER is already set in setUpClass
      0aca21e47a99
  15. 26 Jun, 2017 1 commit
  16. 23 Jun, 2017 2 commits
  17. 09 Jun, 2017 3 commits
  18. 30 May, 2017 1 commit
    • Philippe Pepiot's avatar
      test: run celery monitor after the tasks finish in non EAGER mode · f736030ffc25
      Philippe Pepiot authored
      When running tests in non EAGER mode, tasks workflow synchronization was done
      in run_all_tasks(), this seemed to work because task execution was
      asynchronously finished before call to celery-monitor.
      
      Fix this by explicitely running celery-monitor in wait_async_task which wait
      the task is actually finished.
      f736030ffc25
  19. 19 May, 2017 2 commits