Go Cloud Storage

Open-source Go projects categorized as Cloud Storage

Top 15 Go Cloud Storage Projects

Cloud Storage
  • rclone

    "rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files

    Project mention: Show HN: Horizon – Private alternative to Imgur | news.ycombinator.com | 2024-07-17
  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • Cloudreve

    🌩支持多家云存储的云盘系统 (Self-hosted file management and sharing system, supports multiple storage providers)

  • SFTPGo

    Full-featured and highly configurable SFTP, HTTP/S, FTP/S and WebDAV server - S3, Google Cloud Storage, Azure Blob

    Project mention: What you guys are hosting instead of Nextcloud? I'm sick of it. | /r/selfhosted | 2023-11-29

    EDIT: Thanks for the recommendations from all of you!! I've chose to use the below: - Files: sftpgo - Calendar: baikal - Notes: memos (But beware, it sends opt-out telemetry) - Network folder: webdav on sftpgo

  • goofys

    a high-performance, POSIX-ish Amazon S3 file system written in Go

    Project mention: Is Posix Outdated? | news.ycombinator.com | 2023-10-19

    The author needs to ask themselves: in this cloud technology stack, is there POSIX involved somewhere lower down, where I can't access it? The answer is, of course, "yes". The sort of cloud storage systems described all run on top of POSIX APIs. They provide convenience (cost efficiency is more debatable) compared to the POSIX alternative, but that's because they exist at an entirely different conceptual layer (hence the presence of POSIX anyway, just buried).

    Your point about surfacing a POSIX that's actually there but hidden and thus visible to low-level Amazon employees building the S3 service which makes it invisible to S3 end customers is true but isn't the the point of the article. The author is saying there are motivations for a POSIX-like api visible also the end user.

    So your explanation of stack looks like 2 layers: POSIX api <-- AWS S3 built on top of that

    Author's essay is actually talking about 3 layers: POSIX <-- AWS S3 <-- POSIX

    That's why the blog post has the following links to POSIX-on-top-of-S3-objects :

    https://github.com/s3fs-fuse/s3fs-fuse

    https://github.com/kahing/goofys

    https://www.cuno.io/

  • cubefs

    cloud-native distributed storage

  • google-cloud

    Google Cloud Client Libraries for Go.

    Project mention: Out of process pagination on BigQuery table | /r/bigquery | 2023-12-11

    Hello. So we use big query as database for all events that are happening in the project. And one of the features on the frontend is to display events with detailed view and so. Also one thing that we're using there is infinite scroll, so you can hit and fetch more requests. Based on that I do not know if the current approach is something correct. Basically we need out of process pagination where we create a job on the first request and then on next requests we paginate over the results. Using more or less this solution now: https://github.com/googleapis/google-cloud-go/issues/8173

  • daptin

    Daptin - Backend As A Service - GraphQL/JSON-API Headless CMS

    Project mention: Ask HN: Show me your half baked project | news.ycombinator.com | 2023-10-12
  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
  • gokv

    Simple key-value store abstraction and implementations for Go (Redis, Consul, etcd, bbolt, BadgerDB, LevelDB, Memcached, DynamoDB, S3, PostgreSQL, MongoDB, CockroachDB and many more)

    Project mention: Map of Golang Key Value Engines | news.ycombinator.com | 2024-01-21

    Really great overview!

    I've been tracking some of them for a while as part of evaluating which ones to add to my key-value abstraction library gokv [1], but others only noticed recently. It's really interesting that there's no single most popular implementation, but new ones emerging and gaining popularity regularly.

    [1] https://github.com/philippgille/gokv

  • storage

    A vendor-neutral storage library for Golang: Write once, run on every storage service.

  • zfsbackup-go

    Backup ZFS snapshots to cloud storage such as Google, Amazon, Azure, etc. Built with the enterprise in mind.

  • terraform-provider-iterative

    ☁️ Terraform plugin for machine learning workloads: spot instance recovery & auto-termination | AWS, GCP, Azure, Kubernetes

  • reva

    WebDAV/gRPC/HTTP high performance server to link high level clients to storage backends

  • harp

    Secret management by contract toolchain (by elastic)

  • prvt

    🔐 Personal, end-to-end encrypted storage that works through your browser

  • webdav

    file server written in go. (by rexlx)

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Go Cloud Storage discussion

Log in or Post with

Go Cloud Storage related posts

  • Show HN: Horizon – Private alternative to Imgur

    1 project | news.ycombinator.com | 17 Jul 2024
  • 8 days downtime: Cloudflare r2 subscription bug ruins my business

    1 project | news.ycombinator.com | 16 Jun 2024
  • Ask HN: Best modern file transfer/synchronization protocol?

    5 projects | news.ycombinator.com | 22 Dec 2023
  • Ask HN: How do you do personal backups in 2023? (Google and Dropbox issues)

    1 project | news.ycombinator.com | 11 Dec 2023
  • Which synchronization tool are you using together with the pCloud Crypto Folder?

    2 projects | /r/pCloud_uncensored | 10 Dec 2023
  • Backup of Google Drive (and photos?) to local disk (not to Google Drive)

    1 project | /r/selfhosted | 10 Dec 2023
  • Backuping Google Drive to a Raspberry Pi

    1 project | /r/raspberry_pi | 6 Dec 2023
  • A note from our sponsor - SaaSHub
    www.saashub.com | 21 Jul 2024
    SaaSHub helps you find the best software and product alternatives Learn more →

Index

What are some of the best open-source Cloud Storage projects in Go? This list will help you:

Project Stars
1 rclone 45,223
2 Cloudreve 21,281
3 SFTPGo 8,614
4 goofys 5,115
5 cubefs 4,494
6 google-cloud 3,660
7 daptin 1,812
8 gokv 708
9 storage 536
10 zfsbackup-go 331
11 terraform-provider-iterative 287
12 reva 165
13 harp 145
14 prvt 118
15 webdav 0

Sponsored
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com

Did you konow that Go is
the 4th most popular programming language
based on number of metions?