Open-source projects categorized as data-storage | Edit details

Top 6 data-storage Open-Source Projects

  • react-native-mmkv-storage

    An ultra fast (0.0002s read/write), small & encrypted mobile key-value storage framework for React Native written in C++ using JSI

    Project mention: Why does my RN app seem to speed up as I use it on iOS? | | 2022-04-12

    Use mmkv storage to make it faster

  • knoxite

    A data storage & backup system

  • SonarQube

    Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

  • openHistorian

    The Open Source Time-Series Data Historian

    Project mention: openHistorian: NEW Data - star count:133.0 | | 2021-12-04
  • autotier

    A passthrough FUSE filesystem that intelligently moves files between storage tiers based on frequency of use, file age, and tier fullness.

    Project mention: Any news on Qtier or similar reaching QuTS Hero? | | 2021-08-09

    Given current pricing, larger SSDs, in terms of about 1TB aren't actually twice the price of 512GB ones, which in turn aren't twice as expensive as the 256GB units, seem to be better buys. At the same time, there are benefits to going SSDs for certain capacities and workloads, while in many cases a correctly functioning tiering system is more beneficial. Qnap's Qtier, however, is using dm-thin/md-adm features that are not in use/available by design in QuTS Hero and as such their regular QTS's feature of Qtier is unavailable on the ZFS-based variant. SSDs can be used for SSD pools or caches (as in ZIL for writes/L2ARC for reads/metadata for metadata and deduplication tables or a mix of those) only, similar to QES. Contrary to QES target audience, many ZFS-powered units are reaching customer base that is more home, SOHO or even light on the budget, so going with "huge" (in terms of disk quantity and overall capacity) and resilient all-flash pools is merely a dream, so the caches are more or less the goto option. As stated earlier, this leads to often either overexpenditure or underutilisation of the disk resources. Still, Qnap seems to have been quite tight-lipped about any possibility of Qtier-like solution being offered in their QuTSHero stack for a long time (presumably either because they don't want to offer it to not canibalise any of the remaining comparative QTS only or QES solution sales, or due to their inablity to come up with a solution). Given that the underlying system is a tuned linux, I wonder why autotier-like ( FUSE filesystem has not been prepared as a "replacement"/alternative. Sure, block-level monitoring/tiering would have been even better but the nature of COW/ZFS does not play well with regular block-monitoring tools (one could splice something up but that would require yet another huge table for block counts tracking to combine spinning rust and silicon traps in persistent and ephemeral levels into one global map for storage). However, a FUSE filesystem of autotier deals in file-level analysis, which would greatly increase usefulness of a mixed SSD and HDD pools appliance possibly without incurring much of a performance impact. Qnap certainly has the manpower or funds to prepare a solution that is unique/optimised for their devices that while not as good at moving hot data around as the block-level solution, would be the next best thing without too much of a hustle and it could still joggle around data when required. The question then remains, why haven't Qnap gone through with releasing such a solution, or maybe there is one in the works that Qnap plans to add as a showcase for QuTS Hero v.5? It could greatly improve the QuTS perspective in devices such as TS-673A (wherein one could than get a 3-tier solution with 2x NVMe SSDs for hottest files/data or caches, and the 6 drivebays populated with 3x HDDs for cold and 3x SATA SSDs for warm data layers) or its 8-bay counterpart, or the TS-h973AX (wherein we're getting 3 tiers by design with 2x U.2 for hot sauce or caches + 2x SATA SSDs for warm data + 5x HDDs for cold storage) as example units of somewhat lower pricing in the ZFS-based range/spectrum. This could certainly boost sales of the ZFS units. Or maybe, I've just been deaf and blind to the corporate PR to that degree that I've missed Qnap official unveiling of a future showcase piece plans for later? Or maybe such features have crept silently in some beta releases already and been dropped due to some issues?

  • Java-SerialX

    Store Java objects into file via SerialX. SerialX is a powerful utility library to serialize objects in Java programmatically via recursive descent parser for custom domain-specific languages!

    Project mention: Java SerialX API – Improved serializer and parser for DSL languages like JSON | | 2021-08-22
  • securedata

    A package used in many of my other repos for easily storing data to/from a shared JSON file. Streamlines your workflow for logging and data management.

    Project mention: My weather display (Pi 3B + Inky WHAT) (Instructions in comments) | | 2022-03-21

    You can either modify directly or use securedata, a tool I've recently built to store data across my repos in a common JSON file (yes, the name is kind of misleading)

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2022-04-12.

data-storage related posts


What are some of the best open-source data-storage projects? This list will help you:

Project Stars
1 react-native-mmkv-storage 905
2 knoxite 222
3 openHistorian 141
4 autotier 107
5 Java-SerialX 0
6 securedata 0
Find remote jobs at our new job board There are 7 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives