Python Queue

Open-source Python projects categorized as Queue

Top 23 Python Queue Projects

  • rq

    Simple job queues for Python

    Project mention: The Many Problems with Celery | /r/Python | 2023-05-22 is to the rescue.

  • boltons

    🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.

  • Onboard AI

    Learn any GitHub repo in 59 seconds. Onboard AI learns any GitHub repo in minutes and lets you chat with it to locate functionality, understand different parts, and generate new code. Use it for free at

  • huey

    a little task queue for python

    Project mention: Nextflow: Data-Driven Computational Pipelines | | 2023-08-10

    I've considered using Nextflow for bioinformatics pipelines but have yet to take the plunge. At work, I develop a proteomics pipeline that is composed of huey¹ tasks (Python library; simple alternative to Celery) which either use subprocess to call out to some external tool, or are just pure python. It runs in a worker container which is created by docker swarm, and all containers pull jobs from redis. For our scale, it works great. However, I don't have control over the resource utilization of individual steps, and in the past I've had issues with the pipeline blocking as a result of how I was chaining tasks together. I think something like Nextflow would remove these limitations, but one thing I think I would miss is the ability to debug individual pipeline steps locally with an interactive debugger. As far as I can tell, Nextflow has logging/tracing facilities but nothing quite like an interactive debugger. I'd be happy to be told I'm wrong, or even that I'm doing it wrong.



  • kombu

    Messaging library for Python.

  • arq

    Fast job queuing and RPC in python with asyncio and redis.

    Project mention: The Many Problems with Celery | /r/Python | 2023-05-22
  • tasktiger

    Python task queue using Redis

  • Streamz

    Real-time stream processing for python

    Project mention: Experiences using Streamz? | /r/dataengineering | 2023-02-27

    Hi everyone, I am looking into building data pipelines in a way that is as pythonic as possible, I found out about this Streamz project and it integrates well with Dask. Does anyone have any experience using this? My main use would be for fairly simple accumulator pipelines.

  • InfluxDB

    Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.

  • mrq

    Mr. Queue - A distributed worker task queue in Python using Redis & gevent

  • procrastinate

    PostgreSQL-based Task Queue for Python

    Project mention: Running Procrastinate from command line throwing exception | /r/learnpython | 2023-06-05

    I did find this PR which adds a much more detailed description of what to do, although some of it is a bit outdated.

  • django-background-tasks

    A database-backed work queue for Django

  • KQ

    Kafka-based Job Queue for Python

  • wakaq

    Background task queue for Python backed by Redis, a super minimal Celery

  • django_dramatiq

    A Django app that integrates with Dramatiq.

  • saq

    Simple Async Queues

    Project mention: The Many Problems with Celery | /r/Python | 2023-05-22
  • Flask-RQ2

    A Flask extension for RQ.

  • think-async

    🌿 Exploring cooperative concurrency primitives in Python

  • rele

    Easy to use Google Pub/Sub

  • django-q2

    A multiprocessing distributed task queue for Django. Django Q2 is a fork of Django Q. Big thanks to Ilan Steemers for starting this project. Unfortunately, development has stalled since June 2021. Django Q2 is the new updated version of Django Q, with dependencies updates, docs updates and several bug fixes. Original repository:

    Project mention: django-rq VS django-q2 - a user suggested alternative | | 2023-03-21

    Actively maintained fork of original django-q lib.

  • simpleq

    A simple, infinitely scalable, SQS based queue.

  • youtube-dl-nas

    youtube download queue websocket server with login for private NAS.

  • litequeue

    Queue built on top of SQLite

    Project mention: Choose Postgres Queue Technology | | 2023-09-24

    To make sure you that the message you are trying to retrieve hasn't been locked already by another worker.



  • spinach

    Modern Redis task queue for Python 3 (by NicolasLM)

    Project mention: Django Styleguide | | 2023-01-11

    Since everyone suggests alternatives to Celery, may I plug my own healthier celery?

  • repid

    Repid framework: simple to use, fast to run and extensible to adopt job scheduler

    Project mention: Show HN: Repid – new Python job scheduler with Asyncio in mind | | 2023-03-26
  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-09-24.

Python Queue related posts


What are some of the best open-source Queue projects in Python? This list will help you:

Project Stars
1 rq 9,250
2 boltons 6,296
3 huey 4,706
4 kombu 2,649
5 arq 1,738
6 tasktiger 1,321
7 Streamz 1,197
8 mrq 878
9 procrastinate 615
10 django-background-tasks 577
11 KQ 562
12 wakaq 552
13 django_dramatiq 307
14 saq 306
15 Flask-RQ2 219
16 think-async 217
17 rele 196
18 django-q2 190
19 simpleq 164
20 youtube-dl-nas 146
21 litequeue 116
22 spinach 58
23 repid 50
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives