django版本升级导致celery启动报错,主要原因:django升级后的版本和celery版本不兼容。
我的案例:
# 升级前
django==1.8.11
celery==3.1.18
django-celery==3.2.1
# 升级后
django==1.11.24
celery==3.1.25
django-celery==3.2.2
报错信息:
[2019-12-20 14:44:58,741: ERROR/Worker-1] Process Worker-1
Traceback (most recent call last):
File "C:\Python3.6\lib\site-packages\kombu\utils\__init__.py", line 323, in __get__
return obj.__dict__[self.__name__]
KeyError: 'backend'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Python3.6\lib\site-packages\billiard\process.py", line 292, in _bootstrap
self.run()
File "C:\Python3.6\lib\site-packages\billiard\pool.py", line 292, in run
self.after_fork()
File "C:\Python3.6\lib\site-packages\billiard\pool.py", line 395, in after_fork
self.initializer(*self.initargs)
File "C:\Python3.6\lib\site-packages\celery\concurrency\prefork.py", line 81, in process_initializer
app=app)
File "C:\Python3.6\lib\site-packages\celery\app\trace.py", line 161, in build_tracer
backend = task.backend
File "C:\Python3.6\lib\site-packages\celery\app\task.py", line 919, in backend
return self.app.backend
File "C:\Python3.6\lib\site-packages\kombu\utils\__init__.py", line 325, in __get__
value = obj.__dict__[self.__name__] = self.__get(obj)
File "C:\Python3.6\lib\site-packages\celery\app\base.py", line 625, in backend
return self._get_backend()
File "C:\Python3.6\lib\site-packages\celery\app\base.py", line 443, in _get_backend
self.loader)
File "C:\Python3.6\lib\site-packages\celery\backends\__init__.py", line 63, in get_backend_by_url
return get_backend_cls(backend, loader), url
File "C:\Python3.6\lib\site-packages\celery\backends\__init__.py", line 49, in get_backend_cls
return symbol_by_name(backend, aliases)
File "C:\Python3.6\lib\site-packages\kombu\utils\__init__.py", line 96, in symbol_by_name
module = imp(module_name, package=package, **kwargs)
File "C:\Python3.6\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "C:\Python3.6\lib\site-packages\djcelery\backends\database.py", line 11, in <module>
from ..models import TaskMeta, TaskSetMeta
File "C:\Python3.6\lib\site-packages\djcelery\models.py", line 30, in <module>
class TaskMeta(models.Model):
File "C:\Python3.6\lib\site-packages\django\db\models\base.py", line 110, in __new__
app_config = apps.get_containing_app_config(module)
File "C:\Python3.6\lib\site-packages\django\apps\registry.py", line 247, in get_containing_app_config
self.check_apps_ready()
File "C:\Python3.6\lib\site-packages\django\apps\registry.py", line 125, in check_apps_ready
raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
我的理解是:加载celery任务模块的时候,app没有加载,导致报错。这和我以前遇到的一个案例有点相似,异步周期使用djangorestframe的序列化一样,在import序列化类时,启动celery会报错,序列化加载在前,celery任务加载在后,导致报错。
在这里有两种解决方案:
第一种:在异步周期文件中加入(要求:放在文件内容的最上方)
import django
django.setup()
个人感觉:虽然解决了这种版本不兼容问题,异步周期也都正常能用了,但是在文件最上方引入这些代码,如果下面有其他的导入包,会不符合flake8规则,我个人是不推荐这种用法的,虽然它也解决了这种不兼容
第二种:在异步周期文件中加入
import os
from celery import Celery, platforms
from django.conf import settings
#celery不能root用户启动解决
platforms.C_FORCE_ROOT = True
# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'settings')
# Celery.config_from_object方法是用来加载setting/configuration对象中的配置文件;这些configuration对象可以是setting/configuration 模块,也可以是其他属性的配置
# Celery.config_from_object方法的优先级高,当执行改方法的时候,之前设置的配置文件会被覆盖,所以一些该方法配置之外的配置需要在该方法执行之后再进行配置
app = Celery('proj')
app.config_from_object('django.conf:settings')
# Load task modules from all registered Django app configs.
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
个人理解:这种不兼容的问题主要是celery任务加载在前,app加载在后形成的。那么我只需要让app加载在前,celery任务加载在后就行了。异步周期文件加入以上这些代码,让app加载在celery任务加载形成之前,这样就解决了这种版本不兼容导致的错误。