Top 23 Go S3 Projects
High Performance, Kubernetes Native Object StorageProject mention: Refactoring #4: Using Minio to work with S3 buckets locally | dev.to | 2022-01-09
Minio is an open-source object storage server with an Amazon S3 compatible API. If you’re using Homestead as your working environment, you’re super lucky; Minio is pretty easy to install, barely an inconvenience. Follow the instructions on the Laravel docs above, and you should be ready for the next step.
"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex FilesProject mention: Dual booting now and planning to fully convert with 22.04 LTS release and I have a couple questions | reddit.com/r/Ubuntu | 2022-01-22
* Via 'rclone' - https://rclone.org/ - one way sync client, open source and free
OPS - Build and Run Open Source Unikernels. Quickly and easily build and deploy open source unikernels in tens of seconds. Deploy in any language to any cloud.
SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.Project mention: Updated MinIO NVMe Benchmarks: 2.6Tpbs on Get and 1.6 on Put | news.ycombinator.com | 2022-01-04
This is a fair complaint. :)
For filer metadata, you should just pick the one you are most familiar with.
There is a wiki page for production setup. https://github.com/chrislusf/seaweedfs/wiki/Production-Setup
Highly available Prometheus setup with long term storage capabilities. A CNCF Incubating project.Project mention: Do i really want to mess with OpenTelemetry, or just hook straight into Datadog | reddit.com/r/devops | 2022-01-18
Old company was on a smaller scale than the new one. But still https://thanos.io/
🦄 A modern web client for SFTP, S3, FTP, WebDAV, Git, Minio, LDAP, CalDAV, CardDAV, Mysql, Backblaze, ...Project mention: Fly.io – Free Postgres Databases (and free storage volumes, up to 3GB total) | news.ycombinator.com | 2022-01-20
> If we can sift through 1,000 free users and find one who converts to $25k per month, we're delighted.
Does this ever happen? I was contacted by a F500 who needed some custom version of my open source project (https://github.com/mickael-kerjean/filestash). I'm aiming 2k a month with support and I'm afraid discussion won't move on if it's considered too expensive already. 25k sounds unreal
JuiceFS is a distributed POSIX file system built on top of Redis and S3.Project mention: JuiceFS 1.0-beta1 was released under Apache 2.0 | reddit.com/r/golang | 2022-01-13
a high-performance, POSIX-ish Amazon S3 file system written in GoProject mention: What you gonna add to your selfhost stack this year? | reddit.com/r/selfhosted | 2022-01-02
will probably experiment with https://github.com/kahing/goofys and https://litestream.io/ to make services more easily moved between the devices :) Also, will continue working on https://synpse.net/ to make the operations easier.
Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.
Fully featured and highly configurable SFTP server with optional HTTP, FTP/S and WebDAV support - S3, Google Cloud Storage, Azure BlobProject mention: Remote Backup Solution | reddit.com/r/selfhosted | 2022-01-16
You could also check out SFTPGo
ChubaoFS (abbrev. CBFS) is a cloud native distributed file system and object store.Project mention: How are you managing NFS in K8s world? | reddit.com/r/kubernetes | 2021-07-15
A few weeks ago I took a peek at chubaofs and frankly didn't understand how to install it; but it claims to support RWX.
Archival and Restoration for PostgresProject mention: Optimizing Postgres Queries at Scale | news.ycombinator.com | 2021-12-28
MinIO Client is a replacement for ls, cp, mkdir, diff and rsync commands for filesystems and object storage.
MinIO Client SDK for GoProject mention: Why I cannot access minio via sdk? | reddit.com/r/kubernetes | 2021-11-09
I have installed minio on my cluster. I am making an API call with go to create a bucket. But I don't why is not responding as in here I did the same this example but I am getting the following response:
Parallel S3 and local filesystem execution tool.Project mention: S3/100gbps question | reddit.com/r/aws | 2021-12-05
I like to use https://github.com/peak/s5cmd
Measure Amazon S3's performance from any location.
Tool for easy ClickHouse backup and restore with cloud storages supportProject mention: Backing up Plausible Analytics database | reddit.com/r/Clickhouse | 2021-08-03
I set up Plausible Analytics in my Kubernetes cluster and am trying to figure out how to properly back up and restore the Clickhouse database. I am trying to use https://github.com/AlexAkulov/clickhouse-backup but it only supports table of the MergeTree family. Plausible uses a table named schema_migrations which is of type TinyLog, so it's skipped during backups, making restores useless (because the table is restored empty, when Plausible starts it will try to run all the migrations which will fail because the other tables already exist).
Storage and image processing server written in Go
Helm plugin that allows to set up a chart repository in AWS S3.Project mention: How do you store your helm charts? | reddit.com/r/devops | 2021-07-22
we store helm chart sources in GitHub and Jenkins pushes them to S3 using helm s3 plugin. great as we don't need a service like chartmuseum
A Container Storage Interface for S3Project mention: How to scale nginx pod when pod is mounting a volume | reddit.com/r/kubernetes | 2021-08-29
You could also use the s3fs CSI for your storage. There may be some learning curve to getting it working. My only word of advise is to use the examples in the repo, the README.md is stale. I made some notes here.
A vendor-neutral storage library for Golang: Write once, run on every storage service.Project mention: 2021-16 Weekly Report for go-storage: An application-oriented unified storage layer for Golang. | reddit.com/r/golang | 2021-04-23
More and much easier services support. We have built a code generator to generate code like optional function options. Like go-service-example, we can implement a new service in minutes. For now, we have implemented 10 services and more services are in our roadmap.
Abstract File Storage
rpCheckup is an AWS resource policy security checkup tool that identifies public, external account access, intra-org account access, and private resources.Project mention: AWS GitHub script thoughts | reddit.com/r/hacking | 2021-07-24
What do people think of this? https://github.com/goldfiglabs/rpCheckup I am in no way affiliated with this - frankly I am worried if i use this ^ it might have other access things in my account? But i think all it needs is read only?
S3 FileSystem (fs.FS) implementation (by jszwec)Project mention: AWS S3 FileSystem (io/fs) Implementation for Go1.16 | reddit.com/r/golang | 2021-02-01
S3 Reverse Proxy with GET, PUT and DELETE methods and authentication (OpenID Connect and Basic Auth)Project mention: Help configuring an S3 website to be accessible from on prem only | reddit.com/r/aws | 2021-09-10
Why don't you use just an s3 proxy software like this one? https://oxyno-zeta.github.io/s3-proxy/ There are others
Go S3 related posts
Dual booting now and planning to fully convert with 22.04 LTS release and I have a couple questions
1 project | reddit.com/r/Ubuntu | 22 Jan 2022
Got a new Surface Pro 8 and I've some questions (ex MacOS user)
1 project | reddit.com/r/Surface | 21 Jan 2022
rclone does not resume via sftp?
1 project | reddit.com/r/rclone | 20 Jan 2022
Weekly Coders, Hackers & All Tech related thread - 20/01/2022
1 project | reddit.com/r/india | 20 Jan 2022
50TB of storage, no sync needed
1 project | reddit.com/r/DataHoarder | 18 Jan 2022
Getting Rclone failed Error on Sidenoder
1 project | reddit.com/r/QuestPiracy | 17 Jan 2022
What is the fastest way to transfer 100s of GB from one VM to another VM?
1 project | reddit.com/r/Proxmox | 17 Jan 2022
What are some of the best open-source S3 projects in Go? This list will help you:
|3||Seaweed File System||13,701|
Are you hiring? Post a new remote job listing for free.