|almost 2 years ago||8 days ago|
|Apache License 2.0||MIT License|
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
We haven't tracked posts mentioning Fast-Transformer yet.
Tracking mentions began in Dec 2020.
Doubt about transformers
3 projects | /r/MLQuestions | 26 Dec 2022
The GPT Architecture, on a Napkin
4 projects | news.ycombinator.com | 11 Dec 2022
it is all documented here, in writing and in code https://github.com/lucidrains/x-transformers
you will want to use rotary embeddings, if you do not need length extrapolation
[D] Sudden drop in loss after hours of no improvement - is this a thing?
2 projects | /r/MachineLearning | 30 Jul 2021
The Project - Model: The primary architecture consists of a CNN with a transformer encoder and decoder. At first, I used my implementation of self-attention. Still, due to it not converging, I switched to using x-transformer implementation by lucidrains - as it includes improvements from many papers. The objective is simple; the CNN encoder converts images to a high-level representation; feeds them to the transformer encoder for information flow. Finally, a transformer decoder tries to decode the text character-by-character using autoregressive loss. After two weeks of trying around different things, the training did not converge within the first hour - as this is the usual mark I use to validate if a model is learning or not.
Hacker News top posts: May 9, 2021
4 projects | /r/hackerdigest | 9 May 2021
X-Transformers: A fully-featured transformer with experimental features\ (25 comments)
What are some alternatives?
EasyOCR - Ready-to-use OCR with 80+ supported languages and all popular writing scripts including Latin, Chinese, Arabic, Devanagari, Cyrillic and etc.
reformer-pytorch - Reformer, the efficient Transformer, in Pytorch
TimeSformer-pytorch - Implementation of TimeSformer from Facebook AI, a pure attention-based solution for video classification
Perceiver - Implementation of Perceiver, General Perception with Iterative Attention in TensorFlow
flamingo-pytorch - Implementation of 🦩 Flamingo, state-of-the-art few-shot visual question answering attention net out of Deepmind, in Pytorch
DALLE-pytorch - Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch
memory-efficient-attention-pytorch - Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"
performer-pytorch - An implementation of Performer, a linear attention-based transformer, in Pytorch
Conformer - An implementation of Conformer: Convolution-augmented Transformer for Speech Recognition, a Transformer Variant in TensorFlow/Keras
PaLM-pytorch - Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways
SpecBAS - An enhanced Sinclair BASIC interpreter for modern PCs
euporie - Jupyter notebooks in the terminal