Perceiver VS CrabNet

Swin-Transformer-Object-Detection - This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" on Object Detection and Instance Segmentation.

GAT - Graph Attention Networks (https://arxiv.org/abs/1710.10903)

performer-pytorch - An implementation of Performer, a linear attention-based transformer, in Pytorch

query-selector - LONG-TERM SERIES FORECASTING WITH QUERYSELECTOR – EFFICIENT MODEL OF SPARSEATTENTION

Fast-Transformer - An implementation of Fastformer: Additive Attention Can Be All You Need, a Transformer Variant in TensorFlow

Invariant-Attention - An implementation of Invariant Point Attention from Alphafold 2

deepmind-perceiver - My implementation of DeepMind's Perceiver

mat2vec - Supplementary Materials for Tshitoyan et al. "Unsupervised word embeddings capture latent knowledge from materials science literature", Nature (2019).

TimeSformer-pytorch - Implementation of TimeSformer from Facebook AI, a pure attention-based solution for video classification

ProteinStructurePrediction - Protein structure prediction is the task of predicting the 3-dimensional structure (shape) of a protein given its amino acid sequence and any available supporting information. In this section, we will Install and inspect sidechainnet, a dataset with tools for predicting and inspecting protein structures, complete two simplified implementations of Attention based Networks for predicting protein angles from amino acid sequences, and visualize our predictions along the way.

gato - Unofficial Gato: A Generalist Agent

ViTGAN - A PyTorch implementation of ViTGAN based on paper ViTGAN: Training GANs with Vision Transformers.