SaaSHub helps you find the best software and product alternatives Learn more →
Top 23 Go S3 Projects
-
minio
MinIO is a high-performance, S3 compatible object store, open sourced under GNU AGPLv3 license.
MinIO also switched to AGPLv3 a while back, and they stated that “the AGPL license requires that all software connecting with MinIO be 100% open source for you/your users not to be in violation of the license.”[^1] Since Redis and MinIO are somewhat similar, (Both can be used to store and retrieve data. One uses a custom protocol. The other one uses an S3-compatible API.) Should I assume that this statement also applies to Redis?
[^1]: https://github.com/minio/minio/issues/13308#issuecomment-929...
-
InfluxDB
InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.
-
rclone
"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
Dump PostgreSQL database, compress (gzip) and encrypt (gpg) it, and upload to one or more remote S3-compatible storage (rclone).
-
seaweedfs
SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.
I’m interested in how it is compared to seaweedfs[1], which we use for storing weather data (about 3 PB) for ML training.
[1] https://github.com/seaweedfs/seaweedfs
-
thanos
Highly available Prometheus setup with long term storage capabilities. A CNCF Incubating project.
Project mention: Top 10 Prometheus Alternatives in 2024 [Includes Open-Source] | dev.to | 2024-10-22Thanos is a set of components that can be added to an existing Prometheus deployment to enable long-term storage and global query view.
-
Project mention: A faster way to copy SQLite databases between computers | news.ycombinator.com | 2025-05-01
seems like a new release is being worked on: https://github.com/benbjohnson/litestream/pull/636
-
Object Storage: JuiceFS, Minio
-
SFTPGo
Full-featured and highly configurable SFTP, HTTP/S, FTP/S and WebDAV server - S3, Google Cloud Storage, Azure Blob
[ ] Use a WebDAV server that has a RESTful API: SFTPGo, for example.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
Project mention: Streaming Data from Cloud Storage with Mountpoint for Amazon S3 | dev.to | 2025-02-11
# install goofys\ sudo curl -Lo \\ /usr/local/bin/goofys \\ https://github.com/kahing/goofys/releases/latest/download/goofys\ sudo chmod +x /usr/local/bin/goofys # install mount-s3\ wget \\ https://s3.amazonaws.com/mountpoint-s3-release/latest/x86_64/mount-s3.deb\ sudo dpkg -i mount-s3.deb\ sudo apt-get install -f -y
-
-
Project mention: Syncing AWS S3 Storage Bucket Objects to LocalStack S3 Storage Bucket | dev.to | 2025-03-27
The local storage first approach is more performant as compared to the direct approach since the direct approach relies on xargs to achieve parallel transfers using sub-processes. One can spawn multiple separate sub processes using xargs -P N when N is the number of sub processes required. The sub process parallelism is great for S3 sync as this is an I/O bound task. In terms of speed, using xargs even with tuned parallelism can be slow. If you are looking to speed up syncs, you can use tools such as s5cmd which supports true multithreading. I will sure do a tutorial on s5cmd and do a benchmark to check how it compares to using xargs in parallel mode. The approach in this tutorial was not created with the context of large S3 storage buckets. If you have a tutorial on this please share. Thanks again for reaching to the end of this tutorial. Cheers!
-
On Windows, you can download the ZIP file from the Minio Client Releases page and extract it to a folder of your choice. Then add the binary to your PATH:
-
Project mention: Hacking misconfigured AWS S3 buckets: A complete guide | news.ycombinator.com | 2024-09-09
-
-
docker-volume-backup
Backup Docker volumes locally or to any S3, WebDAV, Azure Blob Storage, Dropbox or SSH compatible storage
I came across this amazing project offen/docker-volume-backup which not only can backup volumes locally on a schedule, it can also gracefully shut down the containers before doing so to make sure there are no unfinished writes. It also supports features like notifications, backup rotation, and compression. Additionally, it's compatible with different storage backends, making it easy to switch strategies—for example, to a remote server. And all of this with a simple Docker container. This is the whole Docker Compose configuration I use for it, not much more than the example from the project's docs:
-
peerdb
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
Project mention: Go channels are bad and you should feel bad (2016) | news.ycombinator.com | 2025-04-13I'm guilty of this too https://github.com/PeerDB-io/peerdb/blob/d36da8bb2f4f6c1c821...
The inner channel is a poor man's future. Came up with this to have lua runtimes be able to process in parallel while maintaining ordering (A B C in, results of A B C out)
-
rill
Rill is a tool for effortlessly transforming data sets into powerful, opinionated dashboards using SQL. BI-as-code.
Rill founder here, I have no comment on the UI similarity :) but I would emphasize our vision is building DuckDB-powered metrics layers and exploratory dashboards -- which we presented at DuckCon #6 last month, PDF below [1] -- and less on notebook style UIs like Hex and Jupyter.
Rill is fully open-source under the Apache license. [2]
[1] https://blobs.duckdb.org/events/duckcon6/mike-driscoll-rill-...
[2] https://github.com/rilldata/rill
-
Project mention: Effortless PostgreSQL backups with a user-friendly web interface | news.ycombinator.com | 2024-09-08
-
clickhouse-backup
Tool for easy backup and restore for ClickHouse® using object storage for backup files.
-
-
-
-
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Go S3 discussion
Go S3 related posts
-
Dropbox will require App Indicator support on Linux
-
A faster way to copy SQLite databases between computers
-
Show HN: Colanode, open-source and local-first Slack and Notion alternative
-
Rclone – open-source, multi threaded, command line tool for sending data
-
Use CLI on any OS to read-write your iPhone folders (without cloud or cable)
-
Syncing AWS S3 Storage Bucket Objects to LocalStack S3 Storage Bucket
-
Google change is breaking some digital photo frames
-
A note from our sponsor - SaaSHub
www.saashub.com | 16 May 2025
Index
What are some of the best open-source S3 projects in Go? This list will help you:
| # | Project | Stars |
|---|---|---|
| 1 | minio | 52,431 |
| 2 | rclone | 50,285 |
| 3 | seaweedfs | 24,296 |
| 4 | thanos | 13,495 |
| 5 | litestream | 11,936 |
| 6 | juicefs | 11,573 |
| 7 | SFTPGo | 10,400 |
| 8 | goofys | 5,326 |
| 9 | wal-g | 3,541 |
| 10 | s5cmd | 3,048 |
| 11 | mc | 3,030 |
| 12 | S3Scanner | 2,754 |
| 13 | minio-go | 2,660 |
| 14 | docker-volume-backup | 2,541 |
| 15 | peerdb | 2,523 |
| 16 | rill | 2,045 |
| 17 | pgbackweb | 1,587 |
| 18 | clickhouse-backup | 1,383 |
| 19 | sneller | 1,035 |
| 20 | s3-benchmark | 851 |
| 21 | ftpserver | 710 |
| 22 | s3manager | 707 |
| 23 | mariadb-operator | 647 |
Sponsored
InfluxDB – Built for High-Performance Time Series Workloads
InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.
www.influxdata.com