1. 02 Dec, 2020 1 commit
    • Nicola Spanti's avatar
      [py] Manage case `result.info` is an exception · c64aa4ca9133
      Nicola Spanti authored
      In a client project, we add the case of `result.info` being
      `UException` (from `cubicweb_nemo.utils`). It resulted in a
      problem, because `result.info` is `True` as boolean, but it has
      no key, so it crashed. This patch does not resolve the real
      problem of having a clean `info` member or knowing why it could
      normally have a different type, but at least it eliminates a
      problem for the final users.
      c64aa4ca9133
  2. 12 Oct, 2020 2 commits
  3. 01 Mar, 2019 1 commit
  4. 28 Feb, 2019 3 commits
    • Philippe Pepiot's avatar
      celery-monitor: requeue all pending tasks · fedf86550807
      Philippe Pepiot authored
      It can occur, for instance in case of "worker lost", that task events are not
      sent in the "MONITOR_KEY" list in redis.
      The proposed solution is to regularly check pending tasks status.
      Since this is almost the same use, this can replace the old "requeue" behavior
      of failed sync_task_state() which was based on a "PENDING_KEY" list in redis.
      
      So, every 10 min, was 1min before, but 10 min seems more reasonable, check all
      pending tasks status within the visibility_timeout, since tasks outside this
      range cannot have state stored in the broker.
      fedf86550807
    • Philippe Pepiot's avatar
      Track worker exited prematurely issues · c6ec94bc9f69
      Philippe Pepiot authored
      In case of exited worker (segfault), AsyncResult.traceback is None and
      information is in AsyncResult.result, record it in the failed transition
      comment.
      
      Add a dedicated test class, this require to run the worker in prefork mode,
      otherwise, in solo mode the information about worker lost isn't tracked.
      Use "import ctypes; ctypes.string_at(0)" to simulate a segfault.
      c6ec94bc9f69
    • Philippe Pepiot's avatar
      d748ec59b5d2
  5. 07 Feb, 2019 1 commit
  6. 05 Feb, 2019 1 commit
    • Philippe Pepiot's avatar
      Do not import test tasks in main test process · 85840e934f6f
      Philippe Pepiot authored
      So the tasks aren't in the scope of the process actually launching tasks.
      Dropping this ensure our code works even when tasks aren't registered, this
      avoid to introduce code requiring the task to be registered (celery exception
      NotRegistered).
      85840e934f6f
  7. 30 Jan, 2019 6 commits
    • Philippe Pepiot's avatar
      Update to celery 4 · 90ab474e75a2
      Philippe Pepiot authored
      This is not backward compatible with celery 3, so pin celery>=4
      
      See http://docs.celeryproject.org/en/4.0/whatsnew-4.0.html for summary of changes
      
      Change attributes from celery.app.config to lower case.
      Drop setting serializer to json in README and tests since json is now the
      default serializer.
      
      I had issues with test_workflow_group() where AsyncResult().get on the *group*
      task_id hang. It looks related to our test infrastructure and AFAIK there is no
      known usage of this for cubes using cubicweb-celerytask.
      I think we must dig into this bug but I already spent enough time on this, so
      let's move forward and skip this part of the test.
      90ab474e75a2
    • Philippe Pepiot's avatar
      Use frozen task id in run_all_tasks() · 09962b2ccb79
      Philippe Pepiot authored
      This code is only used in tests where we check for existence of task_id in database.
      Before running the Task, there is no guarantees that the value of Task.id will
      be used as task_id, this is the purpose of freeze().
      In this particular case Task.id = Task.freeze().id for celery 3.1.25, but this
      is no longer true in celery 4.
      09962b2ccb79
    • Philippe Pepiot's avatar
      Drop get_task_id() · 98bb9bad617b
      Philippe Pepiot authored
      Only Task objects pass through get_task_id() and never AsyncResult, thus get_task_id()
      is now useless and is equivalent to task.freeze().id
      I think an old version of attach_task() was recursively called with either Task
      or AsyncResult thus the handling of both cases, but I cannot find the relevant
      changeset (maybe this code was never used ? ...)
      I'm pretty confident on this because all known use-cases are covered by tests.
      98bb9bad617b
    • Philippe Pepiot's avatar
      Detect celery workflow meta tasks with task.type instead of hasattr · c19d0234c332
      Philippe Pepiot authored
      This make code more readable and less error prone.
      c19d0234c332
    • Philippe Pepiot's avatar
      Use as_tuple() instead of serializable() · 1535d4229beb
      Philippe Pepiot authored
      serializable() and from_serializable() are deprecated in flavor of as_tuple()
      and result_from_tuple().
      This is required to migrate to celery 4
      1535d4229beb
    • Philippe Pepiot's avatar
      flake8 fix over-indented line · 434cc13f80df
      Philippe Pepiot authored
      434cc13f80df
  8. 29 Jan, 2019 1 commit