rocketry
Prefect
rocketry | Prefect | |
---|---|---|
32 | 19 | |
3,178 | 14,645 | |
- | 1.8% | |
0.6 | 10.0 | |
6 months ago | 3 days ago | |
Python | Python | |
MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
rocketry
-
Is SQL a prerequisite to learning Python ?
Intuitive scheduling framework: https://rocketry.readthedocs.io/
-
Easiest way to run a scheduled Python script?
One easy option would be to use Rocketry, it's pure Python:
-
Scheduling in Data Engineering
It has bunch of features: logical scheduling syntax, parallelism/concurrency, dynamic parametization, log to database etc. You can read from the docs: https://rocketry.readthedocs.io/
-
Run external programs intuitively in Python
It's the creator of Rocketry, Red Mail and Red Box again. This week I thought to make it easier to integrate command-line programs to your Python applications.
-
Should i use airflow
If you don't need fancy UIs (or you can create those yourself), I have heard some people are replacing Airflow with Rocketry. Rocketry is very easy to set up and use and its scheduling mechanics are pretty advanced.
-
Email Utility
If you need a scheduler (that also supports custom conditions like "run when there is X amount of emails"), I have another library for that: Rocketry
-
Trying to send Email
Smtplib is a pain to work with (which is the reason I wrote Red Mail). For the scheduling, I also happen to have a library that is often useful for this sorts of problems: https://rocketry.readthedocs.io
-
WHAT ENGINE IS THE BEST FOR PROTOTYPING?(Python)
I have been thinking of creating an algo using Rocketry at some point. It's very easy to create custom conditions and combine these with logical statement. For example, you could set your algo (a function) running when volatility is x amount and you have y amount balance.
-
I have developed a simple Task Orchestrator
I have been tackling the same problem and it has recently gained popularilty. In case you want to take a look at Rocketry: https://github.com/Miksus/rocketry
- A statement-based scheduling framework for Python
Prefect
- Prefect: A workflow orchestration tool for data pipelines
- self hosted Alternative to easycron.com?
-
Example typescript project repos?
If I was answering this question but for python, I'd recommend something like prefect, boto3, or tortoise-orm -- not extremely complex and with a pretty comprehensible featureset.
-
I have developed a simple Task Orchestrator
However, if you are looking for something like this, but much more mature and something of a bloat to be frank, there's Prefect. Honestly, woflo borrows a lot from Prefect conceptually.
-
Dabbling with Dagster vs. Airflow
Disclaimer: I work for Prefect.
It looks like we added cron and other schedule types to the deployment CLI just under a month ago[1].
Over the last couple of releases, we've also made it easier to pull deployments from GitHub or bake your flow code into Docker images instead of needing S3-like storage.
As with any product, there's always more to do, so I appreciate you sharing your thoughts. More than anywhere else I've worked, community feedback is a huge driver of product enhancements and feature development. Feel free to join our Slack community[2] if you'd like to share more feedback or ask questions.
[1] https://github.com/PrefectHQ/prefect/blob/main/RELEASE-NOTES...
- Prefect - The easiest way to automate your data
- Ask HN: Codebases with great, easy to read code?
-
Prefect CLI Action
GitHub Action for running Prefect commands using the Prefect CLI.
- Perfect – Data workflow automation with Python
What are some alternatives?
Python Fire - Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.
dagster - An orchestration platform for the development, production, and observation of data assets.
rocketry-with-fastapi - Example of how to create a scheduler with an API
APScheduler - Task scheduling library for Python
django-formset - The missing widgets and form manipulation library for Django
airbyte - The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
scheduled_thread_pool_executor - Scheduled Thread Pool Executor implementation in python
schedule - Python job scheduling for humans.
protobuf_to_pydantic - Generate a pydantic.BaseModel with parameter verification function from the Python Message object(by the Protobuf file).
doit - task management & automation tool
red-mail - Advanced email sending for Python
django-schedule - A calendaring app for Django. It is now stable, Please feel free to use it now. Active development has been taken over by bartekgorny.