Top 9 Python task-runner Projects

celery

43 23,439 9.5 Python

Distributed Task Queue (development branch)

Project mention: Streaming responses to websockets with multiple LLMs, am I going about this wrong? | /r/LangChain | 2023-06-20

So this might be my understanding, but stuff like celery is more like an orchestrator that chunks up workloads (think Hadoop with multiple nodes).

dramatiq

15 4,051 7.3 Python

A fast and reliable background task processing library for Python 3.

Project mention: Dramatiq: A fast and reliable distributed task processing library for Python | news.ycombinator.com | 2024-02-08

InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
doit

20 1,781 0.0 Python

task management & automation tool

Project mention: How do you deal with CI, project config, etc. falling out of sync across repos? | /r/ExperiencedDevs | 2023-12-06

I like mage for Go and doit for Python.

Poe the Poet

8 1,275 8.1 Python

A task runner that works well with poetry.

Project mention: Template repository for Python projects | /r/Python | 2023-06-08

Compatible with the new pyproject.toml standard, and working well with poetry, there's poethepoet. It's a good alternative, it keeps in the python environment, it's installable through pip(x), and it's cross platform for the windows pythonistas (and there are some, not everyone has more than 1 computer at home...)

pypyr automation task runner

6 568 5.9 Python

pypyr task-runner cli & api for automation pipelines. Automate anything by combining commands, different scripts in different languages & applications into one pipeline process.

Project mention: Simple task runner for automation pipelines | news.ycombinator.com | 2023-11-03

repid

1 58 8.4 Python

Repid framework: simple to use, fast to run and extensible to adopt job scheduler
backstage

7 16 4.3 Python

Three-speed scripting language and task automation tool (by pyrustic)

Project mention: Show HN: Paradict – Streamable multi-format serialization with schema | news.ycombinator.com | 2023-12-18

Although most languages allow Unicode characters in identifiers, for better code portability and readability, we agree to stick to ASCII characters. Since we're already sticking to ASCII characters in our source code, I think we'll be less 'astonished' to encounter similar rules for our configuration keys (especially when a key-value pair in the Paradict configuration file looks like an instruction for variable assignment).
> I don't get the a-z benefit in the argument case - the user must type "arg1" precisely for the argument names to match...
Absolutely ! The user must type "arg1" precisely because this is part of the implicit agreement between the user and the system. If the user forgets to type "arg1", the default value will be taken into account. If the user adds an additional unexpected key (a typo for example), it will be stored in "kwargs" and then ignored or used to warn the user.
I plan to build two flagship projects to leverage Paradict binary and textual formats: a lightweight database and an automation tool. The automation tool will consume a configuration file a bit like another project of mine does (https://github.com/pyrustic/backstage).
And this is where I join you. I think we'll both agree that since a shell command is already likely to have quotes around some of its arguments, it's very annoying to put extra quotes around it. So I'm thinking of introducing a Command data type:
  # typing 'start' in the command line will run

WorkOS

workos.com sponsored

The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
pipelines

3 3 2.7 Python

Create Async Processing Pipelines Quick! (by theboxahaan)
TaskRunner

1 2 6.1 Python

Simple sequential task runner (by hxii)

Project mention: Show HN: I made a simple, sequential task runner | news.ycombinator.com | 2024-01-08

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).