RETRO-pytorch
TorchPQ
| RETRO-pytorch | TorchPQ | |
|---|---|---|
| 2 | 3 | |
| 835 | 203 | |
| - | - | |
| 2.8 | 3.5 | |
| 7 months ago | 6 months ago | |
| Python | Cuda | |
| Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
RETRO-pytorch
-
[D] Any pre trained retrieval based language models available?
There's a Github project that an individual put together based on the RETRO paper. If you checkout the issues list there is some info on work on a pretrained model.
-
[D] Is there an open-source implementation of the Retrieval-Enhanced Transformer (RETRO)?
i'll give it a shot https://github.com/lucidrains/RETRO-pytorch 👍
TorchPQ
-
[D] Is there an open-source implementation of the Retrieval-Enhanced Transformer (RETRO)?
if you're decided to run nearest neighbor search on GPU, you might wanna give TorchPQTorchPQ a try
-
[P] TorchPQ: Efficient Nearest Neighbor Search and Clustering on GPUs
TorchPQ is a python library for approximate nearest neighbor search on GPUs. It has efficient implementations of IVFPQ algorithm as well as some of its variants (e.g IVFPQ+R). The project is written mostly in python using pytorch library, with some custom CUDA kernels to accelerate clustering, searching and indexing.
-
Are there more practical tools for KNN searches and storing documents/embeddings?
If you're using GPU for search, You can give TorchPQ a try, it's an ANN search library implemented with PyTorch and CUDA. with TorchPQ, you can freely add / remove vectors with IDs, you can save/load them just like a regular PyTorch model. TorchPQ is still in development, and I'd love to hear any suggestion or feedback.
What are some alternatives?
CoCa-pytorch - Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch
faiss - A library for efficient similarity search and clustering of dense vectors.
nmslib - Non-Metric Space Library (NMSLIB): An efficient similarity search library and a toolkit for evaluation of k-NN methods for generic non-metric spaces.
deepmind-research - This repository contains implementations and illustrative code to accompany DeepMind publications
FirstCollisionTimestepRarefiedGasSimulator - This simulator computes all possible intersections for a very small timestep for a particle model
DALLE-pytorch - Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch
Scalix - Scalix is a data parallel compute library that automatically scales to the available compute resources.
retomaton - PyTorch code for the RetoMaton paper: "Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval" (ICML 2022)
RetGen
DOKSparse - sparse DOK tensors on GPU, pytorch