Skip to content
  • Philippe Pepiot's avatar
    Migrate task logs from redis and database to logs files · e501a370ed29
    Philippe Pepiot authored
    Having logs stored in redis then in database took too much memory in redis and
    storage in database.
    Using files is far simplier, but it require to have a shared file system (nfs)
    when the worker and the cubiweb instance (the reader) are not in the same
    server.
    
    Use the new cw_celerytask_helpers filelogger instead of redislogger.
    Logs are stored in celerytask-log-dir directory in gzip with a predictible
    filename based on task_id (which is unique).
    
    Drop task_logs attribute from CeleryTask and update tests accordingly.
    celery-monitor don't copy anymore from redis to database when the task is
    ended.
    e501a370ed29