tantivy
paradedb
tantivy | paradedb | |
---|---|---|
48 | 16 | |
9,955 | 3,962 | |
2.2% | 8.7% | |
9.1 | 9.8 | |
7 days ago | 1 day ago | |
Rust | Rust | |
MIT License | GNU Affero General Public License v3.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
tantivy
-
SeekStorm VS tantivy - a user suggested alternative
2 projects | 22 Mar 2024
-
What is Hybrid Search?
Tantivy - a full-text indexing library written in Rust. Has a great performance and featureset.
- Tantivy – Fast, OSS full-text search library in Rust
-
RAG Using Unstructured Data and Role of Knowledge Graphs
By this I presume you mean build a search index that can retrieve results based on keywords? I know certain databases use Lucene to build a keyword-based index on top of unstructured blobs of data. Another alternative is to use Tantivy (https://github.com/quickwit-oss/tantivy), a Rust version of Lucene, if building search indices via Java isn't your cup of tea :)
Both libraries offer multilingual support for keywords, I believe, so that's a benefit to vector search where multilingual embedding models are rather expensive.
-
Show HN: Quickwit – OSS Alternative to Elasticsearch, Splunk, Datadog
We also implemented our schemaless columnar storage optimized for object storage.
The inverted index and columnar storage are part of tantivy [0], which is the fastest search library out there. We maintain it and we decided to build the distributed engine on top of it.
[0] tantivy github repo: https://github.com/quickwit-oss/tantivy
-
Pg_bm25: Elastic-Quality Full Text Search Inside Postgres
The issue for geo search is here: https://github.com/quickwit-oss/tantivy/issues/44
-
Grimoire - A recipe management application.
Search index : Custom-built using tantivy.
-
A Compressed Indexable Bitset
The roaring bitmap variant is used only for the optional index (1 docid => 0 or 1 value) in the columnar storage (DocValues), not for the inverted index. Since this is used for aggregation, some queries may be a full scan.
The inverted index in tantivy uses bitpacked values of 128 elements with a skip index on top.
> I didn't follow the rest of your comment, select is what EF is good at, every other data structure needs a lot more scanning once you land on the right chunk. With BMI2 you can also use the PDEP instruction to accelerate the final select on a 64-bit block
The select for the sparse codec is a [simple array index access](https://github.com/quickwit-oss/tantivy/blob/main/columnar/s...), that is hard to beat. Compression is not good near the 5k threshold though.
-
Job: Rust + Retrieval Systems at Etsy
Hi /r/rust, I’m a SWE on Etsy’s Retrieval Systems team where we’re building a platform based on rust and tantivy (https://github.com/quickwit-oss/tantivy). We’re looking to bring two new engineers onto the team.
-
Announcing Velo - Your Rust-Powered Brainstorming and Note-Taking Tool
Quick Search: Easily find specific notes with Velo's fuzzy-search feature, powered by tantivy. tantivy might have been a little overkill, but it was really easy to integrate.
paradedb
- Using ClickHouse to scale an events engine
-
Code Search Is Hard
Elasticsearch is good, and it does scale, but it is much more cumbersome and expensive to scale and operate than Postgres. If you use the managed service, you'll pay for the operational pain in the form of higher pricing.
The Postgres movement is strong and extensions like ParadeDB https://github.com/paradedb/paradedb are designed specifically to solve this pain point (Disclaimer: I work for ParadeDB)
-
Ask HN: Best way to mirror a Postgres database to parquet?
No timeline yet, but we know it's a high-priority feature and are working hard on it. Best way would be to join our Slack (link here: https://github.com/paradedb/paradedb/blob/dev/README.md) to follow along. It will be in the coming weeks/months, though.
-
Transforming Postgres into a Fast OLAP Database
You're right. We're working on this currently. You can track the issue here: https://github.com/paradedb/paradedb/issues/717
-
We built our customer data warehouse all on Postgres
There are definitely ways to cleanly make Postgres scale for analytics. We didn't discuss in this blog, but we will be writing about them in the future. For example, check out what the folks at ParadeDB are doing. https://github.com/paradedb/paradedb. Neon is doing an awesome job separating compute from storage. Supabase contributed foreign data wrappers make it super easy to read from S3 into Postgres. Lots of great work going out there :)
- Show HN: Pg_analytics – Speed Up Postgres Analytical Queries by 94x
-
Multi-Database Support in DuckDB
Check out https://github.com/paradedb/paradedb/tree/dev/pg_analytics, we're shipping this week
- ParadeDB – PostgreSQL for Search
-
Postgresql index
Shameless plug, but I'm one of the makers of `pg_bm25` (https://github.com/paradedb/paradedb). We're making a faster tsvector/tsrank as a Postgres extension. Maybe it can help, our benchmarks show much faster performance especially as row count increases
- Building an open source vector database. Looking for advice.
What are some alternatives?
sonic - 🦔 Fast, lightweight & schema-less search backend. An alternative to Elasticsearch that runs on a few MBs of RAM.
MeiliSearch - A lightning-fast search API that fits effortlessly into your apps, websites, and workflow
surrealdb - A scalable, distributed, collaborative, document-graph database, for the realtime web
prism - Prism is the easiest way to develop, orchestrate, and execute data pipelines in Python.
milli - Search engine library for Meilisearch ⚡️
retake - PostgreSQL for Search [Moved to: https://github.com/paradedb/paradedb]
pgrx - Build Postgres Extensions with Rust!
quickwit - Cloud-native search engine for observability. An open-source alternative to Datadog, Elasticsearch, Loki, and Tempo.
qdrant - Qdrant - High-performance, massive-scale Vector Database for the next generation of AI. Also available in the cloud https://cloud.qdrant.io/
fselect - Find files with SQL-like queries
bionicgpt - BionicGPT is an on-premise replacement for ChatGPT, offering the advantages of Generative AI while maintaining strict data confidentiality [Moved to: https://github.com/bionic-gpt/bionic-gpt]