Go S3

Open-source Go projects categorized as S3
Edit details
Topics: #Golang #Go #AWS #Storage #Minio
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
Sponsored

Top 23 Go S3 Projects

  • minio

    Multi-Cloud :cloud: Object Storage

    Project mention: has anyone tried integrating govulncheck in their CI workflow? | reddit.com/r/golang | 2023-01-31

    https://github.com/minio/minio/blob/master/.github/workflows/vulncheck.yml works for us.

  • rclone

    "rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex Files

    Project mention: I did it boys: I blew away ~ | reddit.com/r/linux | 2023-02-08

    Rclone

  • InfluxDB

    Build time-series-based applications quickly and at scale.. InfluxDB is the Time Series Platform where developers build real-time applications for analytics, IoT and cloud-native services. Easy to start, it is available in the cloud or on-premises.

  • seaweedfs

    SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

    Project mention: Open source cloud file system. Posix, HDFS and S3 compatible | news.ycombinator.com | 2023-02-08

    Adopted SeaweedFS few months back. Never looked back since then. It's fast even on HDD disks.

    https://github.com/seaweedfs/seaweedfs#introduction

  • thanos

    Highly available Prometheus setup with long term storage capabilities. A CNCF Incubating project.

    Project mention: Prometheus vs EFS: I don't know who to believe | reddit.com/r/sysadmin | 2023-01-24

    You could look at something like Thanos and store your data in S3: https://thanos.io/

  • litestream

    Streaming replication for SQLite.

    Project mention: SQLite-based databases on the Postgres protocol? Yes we can | news.ycombinator.com | 2023-01-25

    - Ben is developing https://github.com/superfly/litefs at Fly after stopping the streaming replication effort https://github.com/benbjohnson/litestream/issues/8

    - And, of course, SQLite has announced a new backend that hopes to support concurrent writes and streaming replication: https://sqlite.org/hctree/doc/hctree/doc/hctree/index.html

    What a time for SQLite

  • juicefs

    JuiceFS is a distributed POSIX file system built on top of Redis and S3.

    Project mention: Open source cloud file system. Posix, HDFS and S3 compatible | news.ycombinator.com | 2023-02-08

    Atomic file/directory renames/moves is the fundamental feature of JuiceFS, which makes it truely a file system rather than a proxy to S3, please check the docs for all the compatibility details [1].

    https://github.com/juicedata/juicefs#posix-compatibility

  • SFTPGo

    Fully featured and highly configurable SFTP server with optional HTTP/S, FTP/S and WebDAV support - S3, Google Cloud Storage, Azure Blob

    Project mention: FTP client lib | reddit.com/r/crystal_programming | 2023-02-06

  • SonarLint

    Clean code begins in your IDE with SonarLint. Up your coding game and discover issues early. SonarLint is a free plugin that helps you find & fix bugs and security issues from the moment you start writing code. Install from your favorite IDE marketplace today.

  • goofys

    a high-performance, POSIX-ish Amazon S3 file system written in Go

    Project mention: Merge my S3 with Mac Finder Folder | reddit.com/r/aws | 2022-11-12

  • wal-g

    Archival and Restoration for databases in the Cloud

    Project mention: WAL-G 2.0: backup and restore for database in the cloud | news.ycombinator.com | 2022-06-23

  • mc

    Simple | Fast tool to manage MinIO clusters :cloud:

    Project mention: Migrating data between two online MinIO instances | dev.to | 2022-07-03

    We are going to use an official MinIO client called mc. I use macOS so installation is quite easy. The process supposes to be similar on different platforms.

  • minio-go

    MinIO Go client SDK for S3 compatible object storage

    Project mention: MinIO - Mock S3 in local development | dev.to | 2022-12-07

    Go

  • s5cmd

    Parallel S3 and local filesystem execution tool.

    Project mention: Those using AWS, have you ever tried to use AWS Transfer Family to transfer files into an S3 bucket? Can I use python to make these uploads, and if so how do I set it up in aws? | reddit.com/r/aws | 2022-10-18

    Some folks say https://github.com/peak/s5cmd is faster than the two options above.

  • clickhouse-backup

    Tool for easy ClickHouse backup and restore with cloud storages support

  • docker-volume-backup

    Backup Docker volumes locally or to any S3, WebDAV, Azure Blob Storage or SSH compatible storage

    Project mention: Online-Speicher als Backup für (teilw. vertrauliche) Dokumente? | reddit.com/r/de_EDV | 2023-01-02

  • s3-benchmark

    Measure Amazon S3's performance from any location.

    Project mention: Ask HN: Have you ever switched cloud? | news.ycombinator.com | 2022-04-08

    There's another benchmark somewhere showing S3 can max out a 100Gbps instance.

    https://github.com/dvassallo/s3-benchmark

    Another potential issue is ListBucket rate limiting. If you have lots of small objects, you'll spend most of the time waiting to discover the names than transferring data

  • csi-s3

    A Container Storage Interface for S3

    Project mention: Learning K3s at Home, troubles with S3 Storage | reddit.com/r/kubernetes | 2022-11-19

    This is the repo I am trying to use to setup s3 storage for config files of services.

  • helm-s3

    ⎈ Helm plugin that allows to set up a chart repository using AWS S3.

  • mort

    Storage and image processing server written in Go

  • Walrus

    🔥 Fast, Secure and Reliable System Backup, Set up in Minutes. (by Clivern)

  • storage

    A vendor-neutral storage library for Golang: Write once, run on every storage service.

  • twitter-clone

    🕊 Twitter Clone developed using Go + Vue 3 + Vite + TailwindCSS + PostgreSQL + Redis (by HotPotatoC)

    Project mention: Clueless as a MoFo | reddit.com/r/WhitePeopleTwitter | 2022-11-10

  • ftpserver

    Golang based autonomous FTP server with SFTP, S3, Dropbox, and Google Drive connectors.

    Project mention: SFTPGo VS ftpserver - a user suggested alternative | libhunt.com/r/sftpgo | 2022-02-10

    Golang based autonomous FTP server with SFTP, S3, Dropbox, and Google Drive connectors.

  • s3manager

    A Web GUI for your S3 buckets

    Project mention: A Web GUI for managing S3 buckets written in Golang !! | reddit.com/r/golang | 2022-08-06

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-02-08.

Go S3 related posts

Index

What are some of the best open-source S3 projects in Go? This list will help you:

Project Stars
1 minio 37,386
2 rclone 36,862
3 seaweedfs 16,462
4 thanos 11,402
5 litestream 8,030
6 juicefs 7,345
7 SFTPGo 5,555
8 goofys 4,501
9 wal-g 2,459
10 mc 2,361
11 minio-go 1,827
12 s5cmd 1,484
13 clickhouse-backup 875
14 docker-volume-backup 765
15 s3-benchmark 687
16 csi-s3 578
17 helm-s3 487
18 mort 478
19 Walrus 452
20 storage 419
21 twitter-clone 391
22 ftpserver 351
23 s3manager 237

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com