rtdl
amplify
| rtdl | amplify | |
|---|---|---|
| 2 | 3 | |
| 43 | 10 | |
| - | - | |
| 0.0 | 7.5 | |
| over 1 year ago | 9 months ago | |
| Go | Go | |
| MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
rtdl
-
Why this subreddit dislikes the so-called Modern Data Stack?
I think my friend and I are building the solution you’re talking about. A real-time data lake that is easy to setup, uses the newest tech (not Spark and Hive), and has an extensible processing layer. https://github.com/realtimedatalake/rtdl. Fingers-crossed, but it looks like we may be a funded startup soon.
-
What is the easiest way to build a real-time data lake?
For context, I work at Segment (including GTM for Segment Data Lakes) and am the co-creator of rtdl (https://github.com/realtimedatalake/rtdl). This is a problem I’m familiar with and trying to solve. I’m really curious about how others make data lakes easier and what solutions are floating around.
amplify
-
Jupyter Lab Extension to run your GPU-heavy stuff (for free for now) on somebody's else server without blocking yours
When using Jupyter Lab and running GPU-heavy notebooks are you annoyed that your computer is not usable for anything else? I made an extension which allows you to run complex AI inference, training,... remotely on decentralized servers [see bacalhau.org]. This allows you to work on multiple GPU-heavy notebooks in parallel. For now Bacalhau is free, so this is a really cool way to run GPU stuff.
- Introducing Bacalhau Amplify: a tool/service that aims to automatically enrich, enhance, and explain your data
-
[P] CleanVision: Audit your Image Data for better Computer Vision
Nice! Thanks for this. Will try this in a project I'm working on: https://github.com/bacalhau-project/amplify/issues/26
What are some alternatives?
incubator-devlake - Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
Dataplane - Dataplane is a data platform that makes it easy to construct a data mesh with automated data pipelines and workflows.
flyte - Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
cleanvision - Automatically find issues in image datasets and practice data-centric computer vision.
Shift - Shift is a high performance better alternative to Airbyte, Singer, Meltano
cleanvision-examples - Notebooks demonstrating example applications of the cleanvision library
deailab - Decentralised AI Jupyter Lab Extension
dbt-core - dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
memphis - Memphis.dev is a highly scalable and effortless data streaming platform
pachyderm - Data-Centric Pipelines and Data Versioning