With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js. Learn more →
Top 23 Storage Open-Source Projects
-
-
spacedrive
Spacedrive is an open source cross-platform file explorer, powered by a virtual distributed filesystem written in Rust.
-
SurveyJS
Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.
-
localForage
💾 Offline storage, improved. Wraps IndexedDB, WebSQL, or localStorage using a simple but powerful API.
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
thanos
Highly available Prometheus setup with long term storage capabilities. A CNCF Incubating project.
-
-
-
react-native-firebase
🔥 A well-tested feature-rich modular Firebase implementation for React Native. Supports both iOS & Android platforms for all Firebase services.
-
-
-
-
-
-
openebs
Most popular & widely deployed Open Source Container Native Storage platform for Stateful Persistent Applications on Kubernetes.
-
-
-
databend
𝗗𝗮𝘁𝗮, 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 & 𝗔𝗜. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
Project mention: A Distributed File System in Go Cut Average Metadata Memory Usage to 100 Bytes | news.ycombinator.com | 2024-02-22Looks like minio added this in 2022:
https://github.com/minio/minio/pull/15433
In the video, Mo talked about a few packages like Cidre and StrOm, and we referred to SpaceDrive.
Not really an answer but have you looked into https://github.com/localForage/localForage
Once the user is logged in, you may need to access information like the user's name, email, etc. Choreo sends this information encoded in a cookie (named userinfo) upon successful login. We can extract the information using a cookie parsing library. For this example, we will be using js-cookie library.
https://github.com/typicode/lowdb is a cool option.
Project mention: Show HN: Insomnium – 100% local and privacy-focus fork of Insomnia API client | news.ycombinator.com | 2023-09-29Yes! I plan to maintain it long-term! I will be rolling out some feature improvements and updates these few weeks.
I still think Kong did a good job in crafting the product. I started using Insomnia in my previous company 3 years ago and our team loved it.
What happened recently felt a little bit like the Unity fiasco (of course, in a much smaller scale). Though as a user I would say Kong had taken a bad turn, I'm still grateful to them for open-sourcing such a fantastic product.
One of the reasons I forked it was because I've read the code and I think it's quite nicely written overall (but unfortunately things like nedb that was used is no longer maintained https://github.com/louischatriot/nedb so it's probably wise to swap out at some point)
curl --silent --remote-name --location https://github.com/ceph/ceph/raw/octopus/src/cephadm/cephadmchmod a+x cephadm./cephadm bootstrap --mon-ip 192.168.1.41
Project mention: State Management Nx React Native/Expo Apps with TanStack Query and Redux | dev.to | 2024-02-17Storage: redux-persist, @react-native-async-storage/async-storage
Project mention: Looking for a way to remote in to K's of raspberry pi's... | /r/sysadmin | 2023-12-10Monitoring = netdata on each RPi https://www.netdata.cloud/ binded to the vpn interface being scraped into a prometeus thaons https://thanos.io/ setup with grafana to give management the Green all is good screens (very important).
Project mention: Show HN: Appendable – Index JSONL data and query via CDN | news.ycombinator.com | 2024-03-05Hi HN! A friend and I were inspired by projects like https://github.com/sql-js/sql.js and the idea of querying files served over CDN with HTTP range requests. We started thinking: what would a database that was specifically designed for this type of use case look like? So we started building one, and we landed on a functional prototype that we're pretty proud of!
With our prototype, Appendable, we're able to serve and query large (GB+) datasets by hosting them on a static file host like Amazon S3 or Cloudflare R2 without running a separate server and worrying about things like tail latency, replication, and connection pooling -- all that is handled for us by the file hoster.
Additionally, one tenet that we have been following is Appendable won't touch your underlying data, so your jsonl file is preserved and we point at that data instead of consuming it into an Appendable-specific file format. This keeps your data yours and makes it easy to introspect the data: just open it up with your favorite editor aka vim.
We're curious what you think, we're excited to build this out further to get the performance even better and add features like pubsub. Everything is open source at https://github.com/kevmo314/appendable.
Kevin and Matthew
I have some experience with Ceph, both for work, and with homelab-y stuff.
First, bear in mind that Ceph is a distributed storage system - so the idea is that you will have multiple nodes.
For learning, you can definitely virtualise it all on a single box - but you'll have a better time with discrete physical machines.
Also, Ceph does prefer physical access to disks (similar to ZFS).
And you do need decent networking connectivity - I think that's the main thing people think of, when they think of high hardware requirements for Ceph. Ideally 10Gbe at the minimum - although more if you want higher performance - there can be a lot of network traffic, particularly with things like backfill. (25Gbps if you can find that gear cheap for homelab - 50Gbps is a technological dead-end. 100Gbps works well).
But honestly, for a homelab, a cheap mini PC or NUC with 10Gbe will work fine, and you should get acceptable performance, and it'll be good for learning.
You can install Ceph directly on bare-metal, or if you want to do the homelab k8s route, you can use Rook (https://rook.io/).
Hope this helps, and good luck! Let me know if you have any other questions.
Project mention: [Demystifying Firebase (Part-1)] How to setup Firebase in React Native CLI App. | dev.to | 2024-04-05We'll be utilizing React Native Firebase for this.
The word on the street is that https://github.com/rhashimoto/wa-sqlite is nearly production ready, closing in on the neat 1.0.0 release, with its IDBBatchAtomic engine highly recommended at https://www.powersync.com/blog/sqlite-persistence-on-the-web. You can try out the benchmark https://rhashimoto.github.io/wa-sqlite/demo/benchmarks.html.
And the other contestant is Dexie, stable https://dexie.org/.
If you were to branch into a new venture today, which one would you pick? And why?
Storing the notes: For storing the notes, we will use the amazing WatermelonDB package which is a popular sqlite wrapper for react-native. Instead of using the default package we will use a fork of this that uses sqlcipher instead of the regular sqlite, allowing us to encrypt the database by passing a secret key.
Project mention: South Korea's No.1 Search Engine Chose JuiceFS over Alluxio for AI Storage | dev.to | 2024-01-18Support for Kerberos keytab files
One note of caution, though: Amplify uses a frontend-only Cognito integration that stores long-lived, never-rotating refresh tokens in browser storage, where any XSS vulnerability would have access to them. A more secure approach is to implement a couple of backend API routes to store the refresh tokens in `HttpOnly` cookies instead, which I outlined here (option 1 in your case to support SSO). I'll probably open source a solution to do this early next year so we don't all have to keep reinventing this wheel (probably why AWS calls their conference re:invent).
Project mention: Here are the 10 projects I am contributing to over the next 6 months. Share yours | dev.to | 2024-04-13WHAT-WG HTML
Project mention: Database Review: Top Five Missing Features from Database APIs | dev.to | 2023-09-14Hasura ❌ (technically yes with Nhost)
Databend vs. Snowflake: https://github.com/datafuselabs/databend/issues/13059
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Storage related posts
- ReductStore CLI Client now in Rust
- Show HN: AutoMQ – A Cost-Effective Kafka distro that can autoscale in seconds
- Show HN: Generate admin apps without writing a single line of code
- Ask HN: Wa-SQLite vs. Dexie, 2024
- ReductStore v1.9.0 Released
- Distribute Artifacts Across OCI Registries
- Add Writingsuggestions="" Attribute
-
A note from our sponsor - SurveyJS
surveyjs.io | 23 Apr 2024
Index
What are some of the best open-source Storage projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | minio | 44,094 |
| 2 | spacedrive | 28,776 |
| 3 | localForage | 24,228 |
| 4 | js-cookie | 21,570 |
| 5 | Lowdb | 20,843 |
| 6 | PouchDB | 16,276 |
| 7 | store.js | 13,977 |
| 8 | NeDB | 13,416 |
| 9 | Ceph | 13,197 |
| 10 | redux-persist | 12,851 |
| 11 | thanos | 12,577 |
| 12 | sql.js | 12,191 |
| 13 | rook | 11,905 |
| 14 | react-native-firebase | 11,410 |
| 15 | Dexie.js | 10,729 |
| 16 | WatermelonDB | 10,074 |
| 17 | juicefs | 9,791 |
| 18 | amplify-js | 9,363 |
| 19 | coursera-dl | 9,249 |
| 20 | openebs | 8,632 |
| 21 | WHATWG HTML Standard | 7,685 |
| 22 | nhost | 7,512 |
| 23 | databend | 7,184 |
Sponsored
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com