Building an efficient sparse keyword index in Python

• NumPy

  272 26,360 10.0 Python

  The fundamental package for scientific computing with Python.

  

Large computations in pure Python can also be painfully slow. Luckily, there is a robust landscape of options for numeric processing. The most popular framework is NumPy. There is also PyTorch and other GPU-based tensor processing frameworks.

• Pytorch

  336 77,783 10.0 Python

  Tensors and Dynamic neural networks in Python with strong GPU acceleration

  

Large computations in pure Python can also be painfully slow. Luckily, there is a robust landscape of options for numeric processing. The most popular framework is NumPy. There is also PyTorch and other GPU-based tensor processing frameworks.

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• txtai

  354 6,953 9.3 Python

  💡 All-in-one open-source embeddings database for semantic search, LLM orchestration and language model workflows

  

Read the full implementation on GitHub to learn more.

• lucene

  11 2,358 9.8 Java

  Apache Lucene open-source search software

  

First, a review of the landscape. As said in the introduction, there aren't a ton of good options. Apache Lucene is by far the best traditional search index from a speed, performance and functionality standpoint. It's the base for Elasticsearch/OpenSearch and many other projects. But it requires Java.

• rank_bm25

  1 845 0.0 Python

  A Collection of BM25 Algorithms in Python

Rank-BM25 project, the top result when searching for python bm25.

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

Suggest a related project