Top 6 Go Bigdata Projects
-
See also: JuiceFS: https://juicefs.com/
Adds a DBMS or key-value store for metadata, making the filesystem much faster (POSIX, small overwrites don't have to replace a full object in the GCS/S3 backend).
Almost certainly a better solution if you want to turn your object storage into a mountable filesystem, with the (big) caveat that you can't access the files directly in the bucket (they are not stored transparently).
-
Project mention: Can we specify nodeSelector inline for a kubectl command | /r/kubernetes | 2022-08-19
Also, if you are creating bare pods, this sounds like batch scheduling and you should consider using Jobs instead, to have a pod controller. And then you could also consider the https://volcano.sh/ scheduler if it has a fitting scheduling plugin for your use case.
-
InfluxDB
Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.
-
-
-
meteor
Meteor is an easy-to-use, plugin-driven metadata collection framework to extract data from different sources and sink to any data catalog. (by odpf)
Project mention: Modern open-source data platform that empowers organizations to discover, transform, analyse and secure data faster and efficiently. | /r/dataengineering | 2022-06-21Meteor is an easy-to-use, plugin-driven metadata collection framework to extract data from different sources and sink to any data catalog.
-
https://github.com/djordje200179/Meduce https://pkg.go.dev/github.com/djordje200179/meduce
Go Bigdata related posts
Index
What are some of the best open-source Bigdata projects in Go? This list will help you:
Project | Stars | |
---|---|---|
1 | juicefs | 8,066 |
2 | volcano | 3,028 |
3 | cds | 926 |
4 | sidekick | 480 |
5 | meteor | 145 |
6 | Meduce | 4 |