Routing-transformer Alternatives

vit-pytorch

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

memory-efficient-attention-pyt

Compact-Transformers

Escaping the Big Data Paradigm with Compact Transformers, 2021 (Train your Vision Transformers in 30 mins on CIFAR-10 with a single GPU!)

tab-transformer-pytorch

Implementation of TabTransformer, attention network for tabular data, in Pytorch

conformer

Implementation of the convolutional module from the Conformer paper, for use in Transformers (by lucidrains)

memory-efficient-attention-pytorch

Discontinued Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"

enformer-pytorch

Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch

LOGICGUIDE

Plug in and Play implementation of "Certified Reasoning with Language Models" that elevates model reasoning by 40%