SaaSHub helps you find the best software and product alternatives Learn more →
Top 23 in-memory Open-Source Projects
-
Typesense
Open Source alternative to Algolia + Pinecone and an Easier-to-Use alternative to ElasticSearch ⚡ 🔍 ✨ Fast, typo tolerant, in-memory fuzzy Search Engine for building delightful search experiences
-
Hazelcast
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
buntdb
BuntDB is an embeddable, in-memory key/value database for Go with custom indexing and geospatial support
-
-
reindexer
Embeddable, in-memory, document-oriented database with a high-level Query builder interface.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
-
-
flashdb
FlashDB is an embeddable, in-memory key/value database in Go (with Redis like commands and super easy to read) (by arriqaaq)
-
-
-
-
-
peaks-consolidation
The Peaks Consolidation is equipped with state-of-the-art algorithms and data structures that support high-performance databending exercises. It specializes in management accounting and consolidation, with some special topics in machine learning and bioinformatics.
-
-
bof-launcher
Beacon Object File (BOF) launcher - library for executing BOF files in C/C++/Zig applications
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
There are actually plenty of non-ES products that are way easier to integrate and tune (and get better results with less effort).
- Typesense (https://github.com/typesense/typesense)
- Algolia
- Google Programmable Search Engine (https://programmablesearchengine.google.com/about/)
Project mention: Does anyone know any good java implementations for distributed key-value store? | /r/ExperiencedDevs | 2023-06-08You're probably looking for Hazelcast here. Note that it does much more than just a distributed k/v, but it will get you where you need to go.
Experimental format to help readability of a long rant:
1.
According to the OP, there's a "terrifying tale of VACUUM in PostgreSQL," dating back to "a historical artifact that traces its roots back to the Berkeley Postgres project." (1986?)
2.
Maybe the whole idea of "use X, it has been battle-tested for [TIME], is robust, all the bugs have been and keep being fixed," etc., should not really be that attractive or realistic for at least a large subset of projects.
3.
In the case of Postgres, on top of piles of "historic code" and cruft, there's the fact that each user of Postgres installs and runs a huge software artifact with hundreds or even thousands of features and dependencies, of which every particular user may only use a tiny subset.
4.
In Kleppmann's DDOA [1], after explaining why the declarative SQL language is "better," he writes: "in databases, declarative query languages like SQL turned out to be much better than imperative query APIs." I find this footnote to the paragraph a bit ironic: "IMS and CODASYL both used imperative query APIs. Applications typically used COBOL code to iterate over records in the database, one record at a time." So, SQL was better than CODASYL and COBOL in a number of ways... big surprise?
Postgres' own PL/pgSQL [2] is a language that (I imagine) most people would rather NOT use: hence a bunch of alternatives, including PL/v8, on its own a huge mass of additional complexity. SQL is definitely "COBOLESQUE" itself.
5.
Could we come up with something more minimal than SQL and looking less like COBOL? (Hopefully also getting rid of ORMs in the process). Also, I have found inspiring to see some people creating databases for themselves. Perhaps not a bad idea for small applications? For instance, I found BuntDB [3], which the developer seems to be using to run his own business [4]. Also, HYTRADBOI? :-) [5].
6.
A usual objection to use anything other than a stablished relational DB is "creating a database is too difficult for the average programmer." How about debugging PostgreSQL issues, developing new storage engines for it, or even building expertise on how to set up the instances properly and keep it alive and performant? Is that easier?
I personally feel more capable of implementing a small, well-tested, problem-specific, small implementation of a B-Tree than learning how to develop Postgres extensions, become an expert in its configuration and internals, or debug its many issues.
Another common opinion is "SQL is easy to use for non-programmers." But every person that knows SQL had to learn it somehow. I'm 100% confident that anyone able to learn SQL should be able to learn a simple, domain-specific, programming language designed for querying DBs. And how many of these people that are not able to program imperatively would be able to read a SQL EXPLAIN output and fix deficient queries? If they can, that supports even more the idea that they should be able to learn something different than SQL.
----
1: https://dataintensive.net/
2: https://www.postgresql.org/docs/7.3/plpgsql-examples.html
3: https://github.com/tidwall/buntdb
4: https://tile38.com/
5: https://www.hytradboi.com/
The article describes that the new JIT is a "copy-and-patch JIT" (I've previously heard this called a "splat JIT"). This is a relatively simple JIT architecture where you have essentially pre-compiled blobs of machine code for each interpreter instruction that you patch immediate arguments into by copying over them.
I once wrote an article about very simple JITs, and the first example in my article uses this style: https://blog.reverberate.org/2012/12/hello-jit-world-joy-of-...
I take some issue with this statement, made later in the article, about the pros/cons vs a "full" JIT:
> The big downside with a “full” JIT is that the process of compiling once into IL and then again into machine code is slow. Not only is it slow, but it is memory intensive.
I used to think this was true also, because my main exposure to JITs was the JVM, which is indeed memory-intensive and slow.
But then in 2013, a miraculous thing happened. LuaJIT 2.0 was released, and it was incredibly fast to JIT compile.
LuaJIT is undoubtedly a "full" JIT compiler. It uses SSA form and performs many optimizations (https://github.com/tarantool/tarantool/wiki/LuaJIT-Optimizat...). And yet feels no more heavyweight than an interpreter when you run it. It does not have any noticeable warm up time, unlike the JVM.
Ever since then, I've rejected the idea that JIT compilers have to be slow and heavyweight.
Project mention: imcache v1.0.0 released. A zero-dependency generic in-memory cache Go library. | /r/golang | 2023-05-03I released first stable version of imcache pkg - https://github.com/erni27/imcache.
Script and Data
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
in-memory related posts
- PostgreSQL: No More Vacuum, No More Bloat
- Does anyone know any good java implementations for distributed key-value store?
- Working on an unfamiliar codebase
- Creating a compiler in Java
- Real-time stream processing with Hazelcast and Pulsar
- Hazelcast - Open-source real-time distributed computation and storage platform
- Hazelcast: Open-source real-time distributed computation and storage platform
-
A note from our sponsor - SaaSHub
www.saashub.com | 24 Apr 2024
Index
What are some of the best open-source in-memory projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | Typesense | 17,876 |
| 2 | Hazelcast | 5,861 |
| 3 | buntdb | 4,381 |
| 4 | tarantool | 3,328 |
| 5 | GCache | 2,510 |
| 6 | memfs | 1,614 |
| 7 | reindexer | 751 |
| 8 | que | 661 |
| 9 | governor | 498 |
| 10 | coherence | 413 |
| 11 | UnityDoorstop | 396 |
| 12 | flashdb | 341 |
| 13 | SwayDB | 288 |
| 14 | opq | 255 |
| 15 | hazelcast-go-client | 184 |
| 16 | hazelcast-nodejs-client | 146 |
| 17 | imcache | 113 |
| 18 | hazelcast-python-client | 113 |
| 19 | zef | 107 |
| 20 | peaks-consolidation | 102 |
| 21 | hazelcast-csharp-client | 101 |
| 22 | bof-launcher | 94 |
| 23 | quickstep | 35 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com