SaaSHub helps you find the best software and product alternatives Learn more →
Top 9 inverted-index Open-Source Projects
-
RediSearch
A query and indexing engine for Redis, providing secondary indexing, full-text search, vector similarity search and aggregations.
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
interpolative_coding
A flexible and efficient C++ implementation of the Binary Interpolative Coding algorithm.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
lofi-dx
A small, fast, local-first, searchable index for client side apps written in Typescript. Supports required, negated, and phrase queries.
-
The EF core algorithm implemented in folly [3] may be a bit faster, and implementing partitioning on top of that is relatively easy.
It would definitely compress much better than roaring bitmaps. In terms of performance, it depends on the access patterns. If very sparse (large jumps) PEF would likely be faster, if dense (visit a large fraction of the bitmap) it'd be slower.
It is possible to squeeze a bit more compression out of PEF by introducing a chunk type for Elias-Fano of the chunk complement (for very dense chunks), but you lose the operation of skipping to a given position, which is however not needed in inverted indexes (you only need to skip past a given id, and that can be supported efficiently). That is not mentioned in the paper because at the time I thought the skip-to-position operation was a non-negotiable.
[1] https://github.com/ot/ds2i/
[2] https://github.com/pisa-engine/pisa
[3] https://github.com/facebook/folly/blob/main/folly/experiment...
Project mention: Show HN: Time Series Benchmark TurboPFor,TurboFloat,TurboFloat LzX,TurboGorilla | news.ycombinator.com | 2023-06-25
The EF core algorithm implemented in folly [3] may be a bit faster, and implementing partitioning on top of that is relatively easy.
It would definitely compress much better than roaring bitmaps. In terms of performance, it depends on the access patterns. If very sparse (large jumps) PEF would likely be faster, if dense (visit a large fraction of the bitmap) it'd be slower.
It is possible to squeeze a bit more compression out of PEF by introducing a chunk type for Elias-Fano of the chunk complement (for very dense chunks), but you lose the operation of skipping to a given position, which is however not needed in inverted indexes (you only need to skip past a given id, and that can be supported efficiently). That is not mentioned in the paper because at the time I thought the skip-to-position operation was a non-negotiable.
[1] https://github.com/ot/ds2i/
[2] https://github.com/pisa-engine/pisa
[3] https://github.com/facebook/folly/blob/main/folly/experiment...
Hey, I tackled phrase matching in my toy project here: https://github.com/vasilionjea/lofi-dx/blob/main/test/search...
I think I tested it thoroughly but any feedback would be appreciated!
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
inverted-index related posts
-
A Compressed Indexable Bitset
-
anyone knows a redis image with redis search and rscoordinator modules loaded?
-
redis cluster is redisearch
-
Simple inverted index, database implemantation
-
Simple inverted index, database implemantation.
-
Build a fuzzy search with PostgreSQL
-
A note from our sponsor - SaaSHub
www.saashub.com | 8 May 2024
Index
What are some of the best open-source inverted-index projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | RediSearch | 5,233 |
| 2 | pisa | 859 |
| 3 | TurboPFor | 745 |
| 4 | MeTA | 684 |
| 5 | ds2i | 141 |
| 6 | postgres-word2vec | 140 |
| 7 | interpolative_coding | 27 |
| 8 | lofi-dx | 7 |
| 9 | database | 7 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com