Scalix
TorchPQ
| Scalix | TorchPQ | |
|---|---|---|
| 1 | 3 | |
| 7 | 203 | |
| - | - | |
| 7.0 | 3.5 | |
| 14 days ago | 5 months ago | |
| Cuda | Cuda | |
| BSD 3-clause "New" or "Revised" License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Scalix
-
Scalix: A Data Parallel Compute Framework w/ Automatic Scaling
The repo can be found here: https://github.com/NAGAGroup/Scalix
TorchPQ
-
[D] Is there an open-source implementation of the Retrieval-Enhanced Transformer (RETRO)?
if you're decided to run nearest neighbor search on GPU, you might wanna give TorchPQTorchPQ a try
-
[P] TorchPQ: Efficient Nearest Neighbor Search and Clustering on GPUs
TorchPQ is a python library for approximate nearest neighbor search on GPUs. It has efficient implementations of IVFPQ algorithm as well as some of its variants (e.g IVFPQ+R). The project is written mostly in python using pytorch library, with some custom CUDA kernels to accelerate clustering, searching and indexing.
-
Are there more practical tools for KNN searches and storing documents/embeddings?
If you're using GPU for search, You can give TorchPQ a try, it's an ANN search library implemented with PyTorch and CUDA. with TorchPQ, you can freely add / remove vectors with IDs, you can save/load them just like a regular PyTorch model. TorchPQ is still in development, and I'd love to hear any suggestion or feedback.
What are some alternatives?
monolish - monolish: MONOlithic LInear equation Solvers for Highly-parallel architecture
faiss - A library for efficient similarity search and clustering of dense vectors.
cub - [ARCHIVED] Cooperative primitives for CUDA C++. See https://github.com/NVIDIA/cccl
RETRO-pytorch - Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch
MegBA - MegBA: A GPU-Based Distributed Library for Large-Scale Bundle Adjustment
nmslib - Non-Metric Space Library (NMSLIB): An efficient similarity search library and a toolkit for evaluation of k-NN methods for generic non-metric spaces.
cugraph - cuGraph - RAPIDS Graph Analytics Library
FirstCollisionTimestepRarefiedGasSimulator - This simulator computes all possible intersections for a very small timestep for a particle model
ArrayFire - ArrayFire: a general purpose GPU library.
deepmind-research - This repository contains implementations and illustrative code to accompany DeepMind publications
instant-ngp - Instant neural graphics primitives: lightning fast NeRF and more
DOKSparse - sparse DOK tensors on GPU, pytorch