Go Bigdata

Open-source Go projects categorized as Bigdata

Top 6 Go Bigdata Projects

  • juicefs

    JuiceFS is a distributed POSIX file system built on top of Redis and S3.

    Project mention: Google Cloud Storage FUSE | news.ycombinator.com | 2023-05-02

    See also: JuiceFS: https://juicefs.com/

    Adds a DBMS or key-value store for metadata, making the filesystem much faster (POSIX, small overwrites don't have to replace a full object in the GCS/S3 backend).

    Almost certainly a better solution if you want to turn your object storage into a mountable filesystem, with the (big) caveat that you can't access the files directly in the bucket (they are not stored transparently).

  • volcano

    A Cloud Native Batch System (Project under CNCF)

    Project mention: Can we specify nodeSelector inline for a kubectl command | /r/kubernetes | 2022-08-19

    Also, if you are creating bare pods, this sounds like batch scheduling and you should consider using Jobs instead, to have a pod controller. And then you could also consider the https://volcano.sh/ scheduler if it has a fitting scheduling plugin for your use case.

  • InfluxDB

    Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.

  • cds

    Data syncing in golang for ClickHouse. (by zeromicro)

  • sidekick

    High Performance HTTP Sidecar Load Balancer (by minio)

  • meteor

    Meteor is an easy-to-use, plugin-driven metadata collection framework to extract data from different sources and sink to any data catalog. (by odpf)

    Project mention: Modern open-source data platform that empowers organizations to discover, transform, analyse and secure data faster and efficiently. | /r/dataengineering | 2022-06-21

    Meteor is an easy-to-use, plugin-driven metadata collection framework to extract data from different sources and sink to any data catalog.

  • Meduce

    MapReduce library for data processing

    Project mention: My MapReduce library | /r/golang | 2023-04-30

    https://github.com/djordje200179/Meduce https://pkg.go.dev/github.com/djordje200179/meduce

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-05-02.

Go Bigdata related posts

Index

What are some of the best open-source Bigdata projects in Go? This list will help you:

Project Stars
1 juicefs 8,066
2 volcano 3,028
3 cds 926
4 sidekick 480
5 meteor 145
6 Meduce 4
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com