SaaSHub helps you find the best software and product alternatives Learn more →
Top 19 object-storage Open-Source Projects
-
seaweedfs
SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.
If you want/need to take out the metadata, there's some nice solutions for that https://github.com/seaweedfs/seaweedfs
-
Backend as a Service (BaaS) goes back to early 2010’s with companies like Parse and Firebase. These products integrated everything a backend provides to a webapp in a single, integrated package that makes it easier to get started and enables you to offload some of the devops maintenance work to someone else.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
Project mention: South Korea's No.1 Search Engine Chose JuiceFS over Alluxio for AI Storage | dev.to | 2024-01-18
Support for Kerberos keytab files
-
databend
𝗗𝗮𝘁𝗮, 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 & 𝗔𝗜. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com
Databend vs. Snowflake: https://github.com/datafuselabs/databend/issues/13059
-
-
# Download the LakeFS binary wget https://github.com/treeverse/lakeFS/releases/latest/download/lakefs # Make the binary executable chmod +x lakefs # Initialize LakeFS with S3 as the storage backend ./lakefs init --backend s3 --s3-gateway-endpoint --s3-region --s3-force-path-style --s3-access-key --s3-secret-key
-
storj
Ongoing Storj v3 development. Decentralized cloud object storage that is affordable, easy to use, private, and secure.
Project mention: Do you donate your CPU time, storage, or bandwidth from your homelab to any altruistic purpose? | /r/homelab | 2023-10-16I did storj.io but was not profitable and the support was worthless. Did join NTP Pool (as I have a stratum 2 GPS NTP) but the power supply died and I haven't been able to get time to fix it.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
S3 Server
Zenko CloudServer, an open-source Node.js implementation of the Amazon S3 protocol on the front-end and backend storage capabilities to multiple clouds, including Azure and Google.
Project mention: Show HN: OpenSign – The open source alternative to DocuSign | news.ycombinator.com | 2023-10-28> Theoretically they could swap with minio but last time we used it it was not a drop-in replacement yet.
Depends on whether AGPL v3 works for you or not (or whether you decide to pay them), I guess: https://min.io/pricing
I've actually been looking for more open alternatives, but haven't found much.
Zenko CloudServer seemed to be somewhat promising, but doesn't seem to be managed very actively: https://github.com/scality/cloudserver/issues/4986 (their Docker images on DockerHub were last updated 10 months ago, which is what the homepage links to; blog doesn't seem active since 2019, forums don't have much going on, despite some action on GitHub still)
There was also Garage, but that one is also AGPL v3: https://garagehq.deuxfleurs.fr/
The closest I got was discovering that SeaweedFS has an S3 compatible mode: https://github.com/seaweedfs/seaweedfs
-
-
-
Project mention: What's the simplest solution to sell digital downloads via SvelteKit? | /r/sveltejs | 2023-12-05
For instance, consider a supabase setup where you place your photos in storage and your orders in a table. Then, have a function run automatically whenever a new row is added to the orders table. That function would be responsible for identifying what the customer bought and delivering it to them.
-
Zenko
Zenko is the open source multi-cloud data controller: own and keep control of your data on any cloud.
> Iteration speed and blazing fast automated tests.
Wholeheartedly agreed!
It is also nice to have that additional assurance of being able to self-host things (if ever necessary) and not being locked into a singular implementation. For example, that's why managed database offerings generally aren't that risky to use, given that they're built on already established projects (e.g. compatible with MySQL/MariaDB/PostgreSQL).
> When I discovered minio, I suddenly got much more confident coding against s3.
MinIO is pretty good, but licensing wise could become problematic if you don't work on something open source but ever want to run it in prod. Not really what this discussion is about, but AGPL is worth mentioning: https://github.com/minio/minio/blob/master/LICENSE
That said, thankfully S3 is so common that we have alternatives even to MinIO available, like Zenko https://www.zenko.io/ and Garage https://garagehq.deuxfleurs.fr/ both of which are good for both local development as well as hosting in whatever environments necessary.
-
garage
(Mirror) S3-compatible object store for small self-hosted geo-distributed deployments. Main repo: https://git.deuxfleurs.fr/Deuxfleurs/garage (by deuxfleurs-org)
Project mention: SeaweedFS fast distributed storage system for blobs, objects, files and datalake | news.ycombinator.com | 2024-02-02Take a look at GarageS3, it's a niceoption for "just an S3 server" for self hosting.
https://garagehq.deuxfleurs.fr/
I use it for self hosting.
-
phalanx
Phalanx is a cloud-native distributed search engine that provides endpoints through gRPC and traditional RESTful API.
-
lithops
A multi-cloud framework for big data analytics and embarrassingly parallel jobs, that provides an universal API for building parallel applications in the cloud ☁️🚀
-
objstore
Go module providing unified interface and efficient clients to work with various object storage providers until like GCS, S3, Azure, SWIFT, COS and more.
-
busyfs
A zero-dependency, single `.h/.a/.so/.dylib/.dll` library to interact (read/write) with ANY object storage, e.g., local filesystem, HDFS, S3, OSS, and more.
Project mention: Show HN: Libobjectfs – zero-dependency, single-artifact library for many storage | news.ycombinator.com | 2023-04-02 -
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
object-storage related posts
- SeaweedFS fast distributed storage system for blobs, objects, files and datalake
- What's the simplest solution to sell digital downloads via SvelteKit?
- A case for moving away from the cloud and embracing local storage solutions
- Show HN: OpenSign – The open source alternative to DocuSign
- Do you donate your CPU time, storage, or bandwidth from your homelab to any altruistic purpose?
- A Step-by-Step Guide to Implementing Data Version Control
- NFT Payload Storage Options
-
A note from our sponsor - SaaSHub
www.saashub.com | 28 Mar 2024
Index
What are some of the best open-source object-storage projects? This list will help you:
Project | Stars | |
---|---|---|
1 | seaweedfs | 20,796 |
2 | parse-server | 20,580 |
3 | juicefs | 9,675 |
4 | databend | 7,078 |
5 | cubefs | 4,280 |
6 | lakeFS | 4,022 |
7 | storj | 2,854 |
8 | S3 Server | 1,617 |
9 | aistore | 1,060 |
10 | Nitrite Database | 797 |
11 | storage | 587 |
12 | Zenko | 535 |
13 | garage | 346 |
14 | phalanx | 341 |
15 | lithops | 296 |
16 | objstore | 80 |
17 | busyfs | 11 |
18 | iceberg-intro-workshop | 10 |
19 | NiFItoKafkaConnect | 3 |