Python Queue

Open-source Python projects categorized as Queue Edit details

Top 20 Python Queue Projects

  • rq

    Simple job queues for Python

    Project mention: What does a "worker" actually look like in an application? | | 2022-07-18

    The search term you're looking for is a "task queue." I'm not very familiar with the Python ecosystem around this, unfortunately, but it sounds like Celery and Redis Queue are both viable options.

  • boltons

    🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.

  • Scout APM

    Less time debugging, more time building. Scout APM allows you to find and fix performance issues with no hassle. Now with error monitoring and external services monitoring, Scout is a developer's best friend when it comes to application development.

  • huey

    a little task queue for python

    Project mention: This Week in Python | | 2022-06-03

    huey – a little task queue for python

  • kombu

    Messaging library for Python.

    Project mention: Variables pool system | | 2022-05-04

    It is feasible and there are a number of ways to do it. There are a few issues, though, mainly how to make sure that you are accessing the latest data and not getting part of the data mid write. A database handles these issues pretty well, since generally they are meant to be an intermediary between different processes. Another solution, that is often more flexible, is a message queue, like this one:

  • arq

    Fast job queuing and RPC in python with asyncio and redis.

    Project mention: I made a simple async queueing framework called SAQ! It includes a built in web UI to manage jobs. | | 2022-01-06

    I need to process a lot of long running IO heavy jobs with background workers. I've been using ARQ for a while but decided to take a crack at writing my own distributed queue.

  • tasktiger

    Python task queue using Redis

    Project mention: This Week In Python | | 2022-07-22

    tasktiger – Python task queue using Redis

  • Streamz

    Real-time stream processing for python

    Project mention: Saturday Daily Thread: Resource Request and Sharing! Daily Thread | | 2022-03-05

    The best candidate so far is 'streamz', which is simple and lays out a processor in a pipeline. But it doesn't quite have the same level as built in monitoring as apache camel, for better or worse.

  • SonarQube

    Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

  • mrq

    Mr. Queue - A distributed worker task queue in Python using Redis & gevent

  • KQ

    Kafka-based Job Queue for Python

  • django-background-tasks

    A database-backed work queue for Django

    Project mention: windows-friendly replacement for django=background-tasks? | | 2022-06-19

    a quick peruse of the issues at django-background-tasks issue board shows, which someone has made a fork that is 4.0 compatible.

  • procrastinate

    PostgreSQL-based Task Queue for Python

    Project mention: Grafana releases OnCall open source project | | 2022-06-14
  • django_dramatiq

    A Django app that integrates with Dramatiq.

  • Flask-RQ2

    A Flask extension for RQ.

    Project mention: Wondering if I should use Celery vs threads for what I want to do | | 2021-08-12

    From experience i would not use threads for this or any background jobs. I would use Celery or Flask-RQ2 to be your workers, you will also probably end up using them to run other tasks as you encounter the need for other jobs. They both use Redis as a broker and job store and you can use Redis for other things like caching and so many other useful features. I kind of like RQ2 more then Celery because its a little simpler but Celery has a lot more to offer, more features. RQ2 has rq-dashboard for monitoring jobs and Celery has Flower.

  • rele

    Easy to use Google Pub/Sub

  • think-async

    🌿 Exploring cooperative concurrency primitives in Python

  • simpleq

    A simple, infinitely scalable, SQS based queue.

  • youtube-dl-nas

    youtube download queue websocket server with login for private NAS.

  • mnqueues

    Monitored Multiprocessing Queues

  • qoo

    A simple library for interacting with Amazon SQS.

  • sendmail-tryqueue

    A generic utility that caches/queues emails that couldn't be sent

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2022-07-22.

Python Queue related posts


What are some of the best open-source Queue projects in Python? This list will help you:

Project Stars
1 rq 8,423
2 boltons 5,842
3 huey 4,170
4 kombu 2,408
5 arq 1,273
6 tasktiger 1,175
7 Streamz 1,079
8 mrq 870
9 KQ 547
10 django-background-tasks 537
11 procrastinate 408
12 django_dramatiq 239
13 Flask-RQ2 202
14 rele 180
15 think-async 178
16 simpleq 161
17 youtube-dl-nas 123
18 mnqueues 2
19 qoo 2
20 sendmail-tryqueue 0
Find remote jobs at our new job board There is 1 new remote job listed recently.
Are you hiring? Post a new remote job listing for free.
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives