[D] Is there an open-source implementation of the Retrieval-Enhanced Transformer (RETRO)?

• deepmind-research

  29 13,298 0.0 Jupyter Notebook

  This repository contains implementations and illustrative code to accompany DeepMind publications

  

  checked their repository and indeed it isn't open sourced yet https://github.com/deepmind/deepmind-research i wouldn't mind taking this on. the only novel thing is the chunked-cross attention (maintaining causality). does your team / company sponsor open-source efforts?

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• RETRO-pytorch

  2 849 2.8 Python

  Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch

  i'll give it a shot https://github.com/lucidrains/RETRO-pytorch 👍

• faiss

  77 31,824 9.6 C++

  A library for efficient similarity search and clustering of dense vectors.

  

  Faisss now includes the 4bit PQ algorithm from scann (https://github.com/facebookresearch/faiss/issues/1399). Looks to me like faiss better maintained and has other options but I don't have any specific experience with these.

• TorchPQ

  3 215 3.5 Cuda

  Approximate nearest neighbor search with product quantization on GPU in pytorch and cuda

  if you're decided to run nearest neighbor search on GPU, you might wanna give TorchPQTorchPQ a try

Suggest a related project