An alternative to Elasticsearch that runs on a few MBs of RAM

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • WorkOS - The modern API for authentication & user identity.
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • LearnThisRepo.com - Learn 300+ open source libraries for free using AI.
  • MeiliSearch

    A lightning-fast search API that fits effortlessly into your apps, websites, and workflow

    Projects like Meili Search are already coming for Elastic Search's lunch: https://www.meilisearch.com. I think there is a market for fast, light weight alternatives like Meili that offers up a fully featured open source experience.

    With Elastic Search many of the features, security being one, are locked away behind commercial licenses. With Meili it seems they are, for the time being anyway, going with a proper open source version. I understand Elastic needs to earn money, and I get their licensing model to accomplish this. But Meili will probably steal away a good portion of customers interested in self hosting their search solution.

  • Typesense

    Open Source alternative to Algolia + Pinecone and an Easier-to-Use alternative to ElasticSearch ⚡ 🔍 ✨ Fast, typo tolerant, in-memory fuzzy Search Engine for building delightful search experiences

    True, but most deployments are also just generic searching of records like Algolia rather than using all the low-level functionality.

    Tyoesense is probably the most compete competitor in that regard: https://typesense.org/

  • WorkOS

    The modern API for authentication & user identity. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  • manticoresearch

    Easy to use open source fast database for search | Good alternative to Elasticsearch now | Drop-in replacement for E in the ELK soon

    > you might look at sphinx search engine

    Manticore Search [1] forked from the latest open source version and has been continually improved for more than 5 years.

    [1] https://manticoresearch.com/

    > although it doesn't has as much features as ES has

    Manticore unlike Sphinx is much closer to Elasticsearch in terms of features set.

  • quickwit

    Cloud-native search engine for observability. An open-source alternative to Datadog, Elasticsearch, Loki, and Tempo.

    Quickwit is a search engine built on top of Tantivy (by the author of Tantivy): https://github.com/quickwit-oss/quickwit

    Quickwit supports Elasticsearch compatible bulk indexing API.

  • sonic

    🦔 Fast, lightweight & schema-less search backend. An alternative to Elasticsearch that runs on a few MBs of RAM.

    - Sonic (https://github.com/valeriansaliou/sonic)

    There isn't enough out there comparing all these for the simple typical fuzzy search/search box usecase -- which I think is 80% of people doing search today.

    Like other people are pointing out, most of these engines won't have all the features of ES (or more accurately Lucene) but I am pretty convinced that most of the time it doesn't actually matter and if someone is searching on your site excessively maybe there's a problem with your UX (unless you're a search engine or repository of information).

    [0]: https://supabase.com/blog/postgres-full-text-search-vs-the-r...

  • Toshi

    A full-text search engine in rust

    You're right I should put bleve on there as well. This isn't even the whole list. Toshi (https://github.com/toshi-search/Toshi) is also out there...

  • tantivy

    Tantivy is a full-text search engine library inspired by Apache Lucene and written in Rust

    My favorite, and the closest one to ElasticSearch (for its features) is probably Tantivy.

    I'd recommend anyone to check up this three projects and choose on what best fits your needs... it's awesome to see that more projects are becoming available by the day!

    [1]: https://github.com/quickwit-oss/tantivy

    [2]: https://github.com/meilisearch/meilisearch

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  • orama

    🌌 Fast, dependency-free, full-text and vector search engine with typo tolerance, filters, facets, stemming, and more. Works with any JavaScript runtime, browser, server, service!

  • OpenSearch

    🔎 Open source distributed and RESTful search engine.

  • zinc

    ZincSearch. A lightweight alternative to elasticsearch that requires minimal resources, written in Go. [Moved to: https://github.com/zinclabs/zinc] (by prabhatsharma)

  • bayard

    A full-text search and indexing server written in Rust.

    Somewhat related, this guy: https://github.com/mosuka/ seems to be very passionate about search service.

    He built two distributed search services:

    - https://github.com/mosuka/phalanx, written in Go.

    - https://github.com/mosuka/bayard, written in Rust.

  • phalanx

    Phalanx is a cloud-native distributed search engine that provides endpoints through gRPC and traditional RESTful API.

    Somewhat related, this guy: https://github.com/mosuka/ seems to be very passionate about search service.

    He built two distributed search services:

    - https://github.com/mosuka/phalanx, written in Go.

    - https://github.com/mosuka/bayard, written in Rust.

  • smyrna

    Prosty konkordancer dla języka polskiego

    I've written a full-text search engine as well. I don't tout it as a replacement for Elasticsearch, but it does have a few advantages: it's fast; supports HTML documents; supports Polish inflection (via a full-blown morphological dictionary, not just a stemmer); and has a very compact on-disk format (pre-parsed HTML trees, Huffman-encoded over large alphabets). Oh, and it's 100% Clojure.

    It underlies a GUI called Smyrna: https://github.com/nathell/smyrna, https://smyrna.danieljanus.pl

    I haven't touched it in six years, other than a few small changes. But I do plan on revisiting it when time permits.

  • search-benchmark-game

    Search engine benchmark (Tantivy, Lucene, PISA, ...)

    This is very very difficult, but Tantivy tried: see https://github.com/quickwit-oss/search-benchmark-game

  • bleve

    A modern text indexing library for go

    I would be interested in such a testbed. I would also like to know how Bleve Search (https://github.com/blevesearch/bleve) turns out.

    I have for many years now a small search engine project in my free-time pipeline, but I'm before crawling even and I intend to sit for searching part after some of that.

  • zincsearch

    ZincSearch . A lightweight alternative to elasticsearch that requires minimal resources, written in Go.

  • OpenSearch-Dashboards

    📊 Open source visualization dashboards for OpenSearch.

  • perftop

    📈 PerfTop: A client for the OpenSearch Performance Analyzer

  • riot

    Go Open Source, Distributed, Simple and efficient Search Engine; Warning: This is V1 and beta version, because of big memory consume, and the V2 will be rewrite all code. (by go-ego)

  • groonga

    An embeddable fulltext search engine. Groonga is the successor project to Senna.

  • Apache Solr

    Apache Lucene and Solr open-source search software

  • nevod

    Nevod is a language and technology for pattern-based text search.

  • Searx

    Privacy-respecting metasearch engine

  • okon

    Fast offline searching for SHA-1 keys in Have I Been Pwned databases

  • algolite

    An Implementation of Algolia to emulate its REST API

  • quickwit

    Quickwit is a fast and cost-efficient distributed search engine for large-scale, immutable data. [Moved to: https://github.com/quickwit-oss/quickwit] (by quickwit-inc)

  • bluge

    indexing library for Go

  • blast

    Blast is a full text search and indexing server, written in Go, built on top of Bleve.

  • docker

    Official docker for Manticore Search (by manticoresoftware)

  • lyra

    🌌 Fast, in-memory, typo-tolerant, full-text search engine written in TypeScript. [Moved to: https://github.com/LyraSearch/lyra] (by nearform)

  • lucene

    Apache Lucene open-source search software

  • solr

    Apache Solr open-source search software

  • konnu

  • beir

    A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.

    There are actually benchmarks that allow measuring search relevancy objectively, e.g. BEIR[1]. Manticore Search team did an effort to make a PR to include it to the list. The results are here [2]. Unfortunately the BEIR team seems to be too busy to review a whole pile of PRs including about Vespa. Nevertheless it would be nice to have both Meilisearch and Typesense there too since it's interesting what performance those non-tf-idf based search engines would show compared to BM25-based and vector search engines.

    [1] https://github.com/beir-cellar/beir

  • go-elasticsearch

    The official Go client for Elasticsearch

  • elasticsearch-py

    Official Python client for Elasticsearch

  • Weaviate

    Weaviate is an open source vector database that stores both objects and vectors, allowing for combining vector search with structured filtering with the fault-tolerance and scalability of a cloud-native database, all accessible through GraphQL, REST, and various language clients.

    + Weaviate for vector based search. Has a BSD-3 license. https://weaviate.io/developers/weaviate/current/

  • Milvus

    A cloud-native vector database, storage for next generation AI applications

  • sanic

    Accelerate your web app development | Build fast. Run fast.

  • LearnThisRepo.com

    Learn 300+ open source libraries for free using AI. LearnThisRepo lets you learn 300+ open source repos including Postgres, Langchain, VS Code, and more by chatting with them using AI!

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts