TorchPQ
Scalix
Our great sponsors
TorchPQ | Scalix | |
---|---|---|
3 | 1 | |
202 | 7 | |
- | - | |
3.5 | 7.5 | |
5 months ago | 10 days ago | |
Cuda | Cuda | |
MIT License | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
TorchPQ
-
[D] Is there an open-source implementation of the Retrieval-Enhanced Transformer (RETRO)?
if you're decided to run nearest neighbor search on GPU, you might wanna give TorchPQTorchPQ a try
-
[P] TorchPQ: Efficient Nearest Neighbor Search and Clustering on GPUs
TorchPQ is a python library for approximate nearest neighbor search on GPUs. It has efficient implementations of IVFPQ algorithm as well as some of its variants (e.g IVFPQ+R). The project is written mostly in python using pytorch library, with some custom CUDA kernels to accelerate clustering, searching and indexing.
-
Are there more practical tools for KNN searches and storing documents/embeddings?
If you're using GPU for search, You can give TorchPQ a try, it's an ANN search library implemented with PyTorch and CUDA. with TorchPQ, you can freely add / remove vectors with IDs, you can save/load them just like a regular PyTorch model. TorchPQ is still in development, and I'd love to hear any suggestion or feedback.
Scalix
-
Scalix: A Data Parallel Compute Framework w/ Automatic Scaling
The repo can be found here: https://github.com/NAGAGroup/Scalix
What are some alternatives?
faiss - A library for efficient similarity search and clustering of dense vectors.
monolish - monolish: MONOlithic LInear equation Solvers for Highly-parallel architecture
RETRO-pytorch - Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch
cub - [ARCHIVED] Cooperative primitives for CUDA C++. See https://github.com/NVIDIA/cccl
nmslib - Non-Metric Space Library (NMSLIB): An efficient similarity search library and a toolkit for evaluation of k-NN methods for generic non-metric spaces.
MegBA - MegBA: A GPU-Based Distributed Library for Large-Scale Bundle Adjustment
deepmind-research - This repository contains implementations and illustrative code to accompany DeepMind publications
cugraph - cuGraph - RAPIDS Graph Analytics Library
FirstCollisionTimestepRarefiedGasSimulator - This simulator computes all possible intersections for a very small timestep for a particle model
ArrayFire - ArrayFire: a general purpose GPU library.
DOKSparse - sparse DOK tensors on GPU, pytorch
instant-ngp - Instant neural graphics primitives: lightning fast NeRF and more