The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning. Learn more โ
Top 9 Python batch-processing Projects
-
pathway
Pathway is a high-throughput, low-latency data processing framework that handles live data & streaming for you. Made with โค๏ธ for Python & ML/AI developers.
-
energy-forecasting
๐ ๐ง๐ต๐ฒ ๐๐๐น๐น ๐ฆ๐๐ฎ๐ฐ๐ธ ๐ณ-๐ฆ๐๐ฒ๐ฝ๐ ๐ ๐๐ข๐ฝ๐ ๐๐ฟ๐ฎ๐บ๐ฒ๐๐ผ๐ฟ๐ธ | ๐๐ฒ๐ฎ๐ฟ๐ป ๐ ๐๐ & ๐ ๐๐ข๐ฝ๐ for free by designing, building and deploying an end-to-end ML batch system ~ ๐ด๐ฐ๐ถ๐ณ๐ค๐ฆ ๐ค๐ฐ๐ฅ๐ฆ + 2.5 ๐ฉ๐ฐ๐ถ๐ณ๐ด ๐ฐ๐ง ๐ณ๐ฆ๐ข๐ฅ๐ช๐ฏ๐จ & ๐ท๐ช๐ฅ๐ฆ๐ฐ ๐ฎ๐ข๐ต๐ฆ๐ณ๐ช๐ข๐ญ๐ด
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
krita-batch-exporter
A Free Krita plugin to batch export layers and groups with maximum flexibility. Scale, trim, export multiple copies of your layers...
-
RevitBatchProcessor
Fully automated batch processing of Revit files with your own Python or Dynamo task scripts!
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
Project mention: Choosing Between a Streaming Database and a Stream Processing Framework in Python | dev.to | 2024-02-10We understood how streaming databases differ from traditional databases, stream processing engines, conventional analytics databases, or OLAP databases. Now letโs focus on when and why we can use stream data processing frameworks for Python as an alternative to streaming databases. Python is the go-to language for data science and machine learning. There are some stream-processing libraries and frameworks in Python such as Bytewax, Quix, GlassFlow, Pathway. They have been developed to cope with the challenges Python Engineers face with Apache Kafka or Flink since they do not natively support Python.
Project mention: Full Stack 7-Steps MLOps Framework โ Learn ML and MLOps by Building an ML System | news.ycombinator.com | 2023-09-26
See https://github.com/pszemraj/textsum. He's the guy that trained most of the popular long finetuned long models. He created a pip package to make life easier (which uses Huggingface under the hood, just pre-selects good models and obfuscates boilerplate).
ExcelToDatabase - Automation tool for batch importing Excel files into MySQL/Oracle/SQL Server/PostgreSQL/Access/Hive/SQLite/Dameng database
Python batch-processing related posts
- Merging data from multiple Excel files
- How to build a custom GPT enabled full-stack app for real-time data
- Show HN: LLM App โ build a realtime LLM app in 30 lines, with no vector database
- Help: Tools > Scripts > Export Layers close when I select "Initial directory"
- ChatGPT in Engineering
- Revit to Navisworks Batch Export - Looking for app or programs to use
-
A note from our sponsor - WorkOS
workos.com | 26 Apr 2024
Index
What are some of the best open-source batch-processing projects in Python? This list will help you:
Project | Stars | |
---|---|---|
1 | pathway | 1,720 |
2 | energy-forecasting | 786 |
3 | krita-batch-exporter | 280 |
4 | RevitBatchProcessor | 268 |
5 | linqit | 245 |
6 | textsum | 110 |
7 | ExcelToDatabase | 46 |
8 | tempren | 7 |
9 | multidim_indexing | 7 |
Sponsored