beneath
optimus
Our great sponsors
| beneath | optimus | |
|---|---|---|
| 2 | 5 | |
| 78 | 737 | |
| - | 0.1% | |
| 0.0 | 4.4 | |
| about 2 years ago | 6 months ago | |
| Go | Go | |
| GNU General Public License v3.0 or later | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
beneath
-
Analyzing the r/wallstreetbets hivemind — August 2021
If you’re interested, here’s the raw Reddit data, my data pipeline, the derived data, and my Jupyter notebook. I’m using Beneath, an open data platform I’m building, to stream and save the data.
-
[Self Promotion] Reddit r/wallstreetbets posts and comments in real-time
The scraper (which uses Async PRAW) is open source here: https://github.com/beneath-hq/beneath/tree/master/examples/reddit
optimus
-
Data Engineering Tools in Go
You can check odpf github, they created some dataops tools using go, one of the example is optimus (https://github.com/odpf/optimus) which is a data pipeline orchestrator
-
Modern open-source data platform that empowers organizations to discover, transform, analyse and secure data faster and efficiently.
Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.
-
Seeking opinions on workflow/DAG orchestration with Go
this may help: https://github.com/odpf/optimus
- Optimus - An easy-to-use, reliable and performant workflow orchestrator for data transformation, data modeling, pipelines and data quality management
- GitHub - odpf/optimus: Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.
What are some alternatives?
Activeloop Hub - Data Lake for Deep Learning. Build, manage, query, version, & visualize datasets. Stream data real-time to PyTorch/TensorFlow. https://activeloop.ai [Moved to: https://github.com/activeloopai/deeplake]
cadence - Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.
whylogs - An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈
Benthos - Fancy stream processing made operationally mundane
pachyderm - Data-Centric Pipelines and Data Versioning
goflow - A Golang based high performance, scalable and distributed workflow framework
sayn - Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).
protobuf-bigquery-go - Seamlessly save and load protocol buffers to and from BigQuery using Go.
flyte - Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
dagu - Yet another cron alternative with a Web UI, but with much more capabilities. It aims to solve greater problems.
oomstore - Lightweight and Fast Feature Store Powered by Go (and Rust).