How to create Periodic Tasks in Celery?

Celery is a powerful, production-ready asynchronous job queue, which allows you to run time-consuming Python functions in the background. Celery is on the Python Package Index (PyPi), and can be easily installed with pip or easy_install and its dependencies.

Installation:

$ pip install django-celery && pip install redis

Celery uses a broker to pass messages between your application and Celery worker processes. We are having like rabbitmq, redis but we are using redis as a broker in this article. To initiate a task, an application will adds a message to the queue, which the broker then delivers to a worker.

So we can install redis using the following command:;

sudo apt-get install redis-server

Setting up celery in your project:

1. We need to add ‘djcelery’ into your project Installed Apps.

INSTALLED_APPS += (
'djcelery',
)

2. To configure celery, we are using redis broker which is easy to install, lightweight, fast:

import djcelery
djcelery.setup_loader()
BROKER_URL = 'redis://localhost:6379/1'

3. We need to tell celery from where tasks are being loaded using CELERY_IMPORTS command

CELERY_IMPORTS = ("testapp.tasks")

4. Celery Periodic Task means which runs at a regular intervals of time. These periodic tasks are scheduled by a celery beat which will executed by a worker.

  • You can set the interval of time using crontab, timedelta. Using the time delta, we can execute a task at particular point of time like
timedelta(seconds=30)     timedelta(minutes=30),     timedelta(minutes=*/5) --> which will execute at every 5 minutes
  • Like with cron, the tasks may overlap if the first task does not complete before the next.
crontab(0, 0, day_of_month='23') --> which will execute on the 23rd day of the every month.     crontab(minute='*/5') --> which will execute at every 5 minutes     crontab(hour='*', minute='5', day_of_week='mon') --> which will execute for every 5 minutes on mon

5. We can setup celery periodic tasks settings using celery beat schedular.

from celery.schedules import crontab    CELERY_TIMEZONE = "Asia/Calcutta"    CELERYBEAT_SCHEDULER = "djcelery.schedulers.DatabaseScheduler"    CELERYBEAT_SCHEDULE = {
# Executes for every 5 minutes on mon,tue,wed,thu,fri,sat
'add-every-day-evening': {
'task': 'testapp.tasks.load_messages_into_database',
'schedule': crontab(hour='*', minute='5', day_of_week='mon,tue,wed,thu,fri,sat'),
},
}

You have to ensure only a single scheduler is running for a schedule at a time, otherwise you would end up with duplicate tasks.

Python, Django, Android and IOS, reactjs, react-native, AWS, Salesforce consulting & development company