Go google-cloud-storage

Open-source Go projects categorized as google-cloud-storage
Edit details
Topics: Golang Go Gcs HacktoberFest S3
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
Sponsored

Top 13 Go google-cloud-storage Projects

  • rclone

    "rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex Files

    • Project mention: Supabase Storage: now supports the S3 protocol | dev.to | 2024-04-18

    rclone: a command-line program to manage files on cloud storage.

  • migrate

    Database migrations. CLI and Golang library.

    • Project mention: Using migrations with Golang | dev.to | 2024-04-17

    Go does not natively support the use of migrations, but we could use the ORM that has this functionality, such as GORM which is the most used by the community, but We can use migrations without using an ORM, for this we will use the golang-migrate package.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • thanos

    Highly available Prometheus setup with long term storage capabilities. A CNCF Incubating project.

    • Project mention: Looking for a way to remote in to K's of raspberry pi's... | /r/sysadmin | 2023-12-10

    Monitoring = netdata on each RPi https://www.netdata.cloud/ binded to the vpn interface being scraped into a prometeus thaons https://thanos.io/ setup with grafana to give management the Green all is good screens (very important).

  • SFTPGo

    Fully featured and highly configurable SFTP server with optional HTTP/S, FTP/S and WebDAV support - S3, Google Cloud Storage, Azure Blob

    • Project mention: What you guys are hosting instead of Nextcloud? I'm sick of it. | /r/selfhosted | 2023-11-29

    EDIT: Thanks for the recommendations from all of you!! I've chose to use the below: - Files: sftpgo - Calendar: baikal - Notes: memos (But beware, it sends opt-out telemetry) - Network folder: webdav on sftpgo

  • kopia

    Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.

    • Project mention: DwarFS – The Deduplicating Warp-Speed Advanced Read-Only File System | news.ycombinator.com | 2024-04-11

    I think Kopia would be great for your use case

    https://kopia.io/

    It has a great system to snapshot files but only store data if it's changed. I use it in an environment where I can't use something like zfs to snapshot data because I don't have the ability to make decisions about what filesystem we're using. It's been amazing, love it so much!

  • goofys

    a high-performance, POSIX-ish Amazon S3 file system written in Go

    • Project mention: Is Posix Outdated? | news.ycombinator.com | 2023-10-19

    The author needs to ask themselves: in this cloud technology stack, is there POSIX involved somewhere lower down, where I can't access it? The answer is, of course, "yes". The sort of cloud storage systems described all run on top of POSIX APIs. They provide convenience (cost efficiency is more debatable) compared to the POSIX alternative, but that's because they exist at an entirely different conceptual layer (hence the presence of POSIX anyway, just buried).

    Your point about surfacing a POSIX that's actually there but hidden and thus visible to low-level Amazon employees building the S3 service which makes it invisible to S3 end customers is true but isn't the the point of the article. The author is saying there are motivations for a POSIX-like api visible also the end user.

    So your explanation of stack looks like 2 layers: POSIX api <-- AWS S3 built on top of that

    Author's essay is actually talking about 3 layers: POSIX <-- AWS S3 <-- POSIX

    That's why the blog post has the following links to POSIX-on-top-of-S3-objects :

    https://github.com/s3fs-fuse/s3fs-fuse

    https://github.com/kahing/goofys

    https://www.cuno.io/

  • lakeFS

    lakeFS - Data version control for your data lake | Git for data

    • Project mention: A Step-by-Step Guide to Implementing Data Version Control | dev.to | 2023-09-04

    # Download the LakeFS binary wget https://github.com/treeverse/lakeFS/releases/latest/download/lakefs # Make the binary executable chmod +x lakefs # Initialize LakeFS with S3 as the storage backend ./lakefs init --backend s3 --s3-gateway-endpoint --s3-region --s3-force-path-style --s3-access-key --s3-secret-key

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo

  • berglas

    A tool for managing secrets on Google Cloud

    • Project mention: How to deploy a Django app to Google Cloud Run using Terraform | dev.to | 2024-01-01

    Secret Manager: secure storage for sensitive data e.g passwords.

  • fake-gcs-server

    Google Cloud Storage emulator & testing library.

    • Project mention: Show HN: Winglang – a new Cloud-Oriented programming language | news.ycombinator.com | 2023-12-06

    > not really.. as soon as you need cloud resources that are not in your k8s cluster, you end up with the cloud in your development loop.

    https://github.com/localstack/awscli-local

    https://github.com/fsouza/fake-gcs-server

    https://cloud.google.com/sdk/docs/downloads-docker

    I’ve used all of these locally to great success.

    Glad to hear you guys want to interop with npm. Is wing going to be a superset of js then?

  • helm-gcs

    Manage Helm 3 repositories on Google Cloud Storage 🔐 **privately**

  • csi-gcs

    Kubernetes CSI driver for Google Cloud Storage

    • Project mention: Google Cloud Storage FUSE | news.ycombinator.com | 2023-05-02

    You're right their Apache licenses are different:

    https://github.com/ofek/csi-gcs/blob/master/LICENSE-APACHE

    https://github.com/GoogleCloudPlatform/gcs-fuse-csi-driver/b...

    OP should submit a PR to correct this. IANAL but pretty sure they're supposed to use the original copy including copyright notice "Copyright 2020 Ofek Lev"

  • Google-Cloud-Guide

    Google Cloud Platform (GCP) Guide. Learn all about Google Cloud Tools, Services, and Certifications.

  • gcs-sftp-server

    SFTP Server with backing storage via Google Cloud Storage (GCS)

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Go google-cloud-storage related posts

Index

What are some of the best open-source google-cloud-storage projects in Go? This list will help you:

Project Stars
1 rclone 43,720
2 migrate 13,946
3 thanos 12,577
4 SFTPGo 8,117
5 kopia 6,270
6 goofys 5,031
7 lakeFS 4,058
8 berglas 1,224
9 fake-gcs-server 965
10 helm-gcs 277
11 csi-gcs 152
12 Google-Cloud-Guide 124
13 gcs-sftp-server 11

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com