Celery tasks not works with gevent

Posted by Novarg on Stack Overflow See other posts from Stack Overflow or by Novarg
Published on 2013-10-21T15:36:11Z Indexed on 2013/10/21 15:53 UTC
Read the original article Hit count: 319

Filed under:
|
|
|
|

When i use celery + gevent for tasks that uses subprocess module i'm getting following stacktrace:

Traceback (most recent call last):
  File "/home/venv/admin/lib/python2.7/site-packages/celery/task/trace.py", line 228, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/venv/admin/lib/python2.7/site-packages/celery/task/trace.py", line 415, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/webapp/admin/webadmin/apps/loggingquarantine/tasks.py", line 107, in release_mail_task
    res = call_external_script(popen_obj.communicate)
  File "/home/webapp/admin/webadmin/apps/core/helpers.py", line 42, in call_external_script
    return func_to_call(*args, **kwargs)
  File "/usr/lib64/python2.7/subprocess.py", line 740, in communicate
    return self._communicate(input)
  File "/usr/lib64/python2.7/subprocess.py", line 1257, in _communicate
    stdout, stderr = self._communicate_with_poll(input)
  File "/usr/lib64/python2.7/subprocess.py", line 1287, in _communicate_with_poll
    poller = select.poll()
AttributeError: 'module' object has no attribute 'poll'

My manage.py looks following (doing monkeypatch there):

#!/usr/bin/env python
from gevent import monkey
import sys
import os

if __name__ == "__main__":
    if not 'celery' in sys.argv:
        monkey.patch_all()
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webadmin.settings")
    from django.core.management import execute_from_command_line
    sys.path.append(".")
    execute_from_command_line(sys.argv)

Is there a reason why celery tasks act like it wasn't patched properly?

p.s. strange thing that my local setup on Macos works fine while i getting such exceptions under Centos (all package versions are the same, init and config scripts too)

© Stack Overflow or respective owner

Related posts about python

Related posts about django