|Go||Seaweed File System|
|5 days ago||4 days ago|
|MIT License||Apache License 2.0|
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
7 subjects (and GitHub repositories) to become a better Go Developer
16 projects | dev.to | 11 Jul 2021
View on GitHub
Gopher Gold #15 - Wed Oct 14 2020
16 projects | dev.to | 14 Oct 2020
TheAlgorithms/Go (Go): Algorithms Implemented in GoLang
Seaweed File System
Updated MinIO NVMe Benchmarks: 2.6Tpbs on Get and 1.6 on Put
3 projects | news.ycombinator.com | 4 Jan 2022
For computers, batch IO operations are much faster than random IO and can easily saturate the network.
This benchmark uses large batch size, 64MB, to test. There is nothing new here. Most common file systems can easily do the same.
The difficult task is to read and write lots of small files. There is a term for it, LOSF. I work on SeaweedFS, https://github.com/chrislusf/seaweedfs , which is designed to handle LOSF. And of course, no problem with large files at all.3 projects | news.ycombinator.com | 4 Jan 2022
This is a fair complaint. :)
For filer metadata, you should just pick the one you are most familiar with.
There is a wiki page for production setup. https://github.com/chrislusf/seaweedfs/wiki/Production-Setup
Is it a good idea to store images in a SQL-like database?
1 project | reddit.com/r/Database | 28 Dec 2021
You can use SeaweedFS, https://github.com/chrislusf/seaweedfs , which just append images to large files. This basically merge many images files together. You can store the image reference id in the database.
Searching for a Unicorn-like Storage Solution
4 projects | reddit.com/r/selfhosted | 20 Dec 2021
Hello, check SeaweedFS
Using Nginx as an Object Storage Gateway
3 projects | news.ycombinator.com | 2 Dec 2021
Interesting that SeaweedFS also has a similar named "Gateway to Remote Object Storage" https://github.com/chrislusf/seaweedfs/wiki/Gateway-to-Remot...
The difference is that SeaweedFS can support both read and write, with asynchronous write back. Nginx can support read only caching with 1 hour TTL
SeaweedFS uses skip list on Redis to store millions of files in one directory.
1 project | reddit.com/r/filesystems | 11 Oct 2021
Will Cloudflare R2 Win Customers from Amazon S3?
2 projects | news.ycombinator.com | 6 Oct 2021
There are simply too many object store vendors, with different storage pricing, egress pricing, network speed, and reliabilities.
What about using SeaweedFS gateway to remote object store? https://github.com/chrislusf/seaweedfs/wiki/Gateway-to-Remot...
Basically you can think it is a cache. You operate with normal S3 APIs, and the data will be uploaded to remote storage asynchronously (most vendors have free uploading).
It's not a magic pill for all users, but it should be very fast, and cut down the API costs, egress costs.
Cloudflare R2 Storage: Rapid and Reliable Object Storage, Minus the Egress Fees
2 projects | news.ycombinator.com | 28 Sep 2021
Most cloud object storage can be a good off-site backup. But accessing your data fast and cheap is not easy.
That is why SeaweedFS added a gateway to remote object store. https://github.com/chrislusf/seaweedfs/wiki/Gateway-to-Remot... , which asynchronously write local changes to the cloud. If there are enough local capacity, there should be no egress cost.
Hopefully, this can change the common pattern, to really treat the cloud object store as a backup.
Clustered filesystem between servers
1 project | reddit.com/r/selfhosted | 27 Sep 2021
I’m going to use https://github.com/chrislusf/seaweedfs on my NAS.
Software projects need heroes? lessons learned from 1100 projects
1 project | news.ycombinator.com | 21 Sep 2021
As a main contributor to an open source project (https://github.com/chrislusf/seaweedfs), I can confirm that this finding is so true.
However, seems this research did not look into Apache projects, which basically maintain a different culture to encourage more contributors, so much as to as the main contributors to refrain from jumping to solve an issue until another person steps in first.
What are some alternatives?
Ceph - Ceph is a distributed object, block, and file storage platform
minio - High Performance, Kubernetes Native Object Storage
GlusterFS - Web Content for gluster.org -- Deprecated as of September 2017
Apache Hadoop - Apache Hadoop
MooseFS - MooseFS – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System (Software-Defined Storage)
lizardfs - LizardFS is an Open Source Distributed File System licensed under GPLv3.
Go IPFS - IPFS implementation in Go
autotier - A passthrough FUSE filesystem that intelligently moves files between storage tiers based on frequency of use, file age, and tier fullness.
Tahoe-LAFS - The Tahoe-LAFS decentralized secure filesystem.
Alluxio (formerly Tachyon) - Alluxio, data orchestration for analytics and machine learning in the cloud
GlusterFS - Gluster Filesystem : Build your distributed storage in minutes
Camlistore - Perkeep (née Camlistore) is your personal storage system for life: a way of storing, syncing, sharing, modelling and backing up content.