Top 6 Go Bigdata Projects
JuiceFS is a distributed POSIX file system built on top of Redis and S3.Project mention: Google Cloud Storage FUSE | news.ycombinator.com | 2023-05-02
See also: JuiceFS: https://juicefs.com/
Adds a DBMS or key-value store for metadata, making the filesystem much faster (POSIX, small overwrites don't have to replace a full object in the GCS/S3 backend).
Almost certainly a better solution if you want to turn your object storage into a mountable filesystem, with the (big) caveat that you can't access the files directly in the bucket (they are not stored transparently).
A Cloud Native Batch System (Project under CNCF)Project mention: Can we specify nodeSelector inline for a kubectl command | /r/kubernetes | 2022-08-19
Also, if you are creating bare pods, this sounds like batch scheduling and you should consider using Jobs instead, to have a pod controller. And then you could also consider the https://volcano.sh/ scheduler if it has a fitting scheduling plugin for your use case.
Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.
Data syncing in golang for ClickHouse. (by zeromicro)
High Performance HTTP Sidecar Load Balancer (by minio)
Meteor is an easy-to-use, plugin-driven metadata collection framework to extract data from different sources and sink to any data catalog. (by odpf)Project mention: Modern open-source data platform that empowers organizations to discover, transform, analyse and secure data faster and efficiently. | /r/dataengineering | 2022-06-21
Meteor is an easy-to-use, plugin-driven metadata collection framework to extract data from different sources and sink to any data catalog.
MapReduce library for data processingProject mention: My MapReduce library | /r/golang | 2023-04-30
Go Bigdata related posts
Can we specify nodeSelector inline for a kubectl command
2 projects | /r/kubernetes | 19 Aug 2022
What are some of the best open-source Bigdata projects in Go? This list will help you: