Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →
Top 23 attention Open-Source Projects
-
nn
🧑🏫 60 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
attention-is-all-you-need-pytorch
A PyTorch implementation of the Transformer model in "Attention is All You Need".
Project mention: ElevenLabs Launches Voice Translation Tool to Break Down Language Barriers | news.ycombinator.com | 2023-10-10The transformer model was invented to attend to context over the entire sequence length. Look at how the original authors used the Transformer for NMT in the original Vaswani et al publication. https://github.com/jadore801120/attention-is-all-you-need-py...
-
pytorch-seq2seq
Tutorials on implementing a few sequence-to-sequence (seq2seq) models with PyTorch and TorchText.
-
-
pytorch-GAT
My implementation of the original GAT paper (Veličković et al.). I've additionally included the playground.py file for visualizing the Cora dataset, GAT embeddings, an attention mechanism, and entropy histograms. I've supported both Cora (transductive) and PPI (inductive) examples!
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
-
-
graphtransformer
Graph Transformer Architecture. Source code for "A Generalization of Transformer Networks to Graphs", DLG-AAAI'21.
-
Project mention: The Secret Sauce behind 100K context window in LLMs: all tricks in one place | news.ycombinator.com | 2023-06-17
-
LongNet
Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
If you want to talk immature looking, longnet wouldn't even compile. That's a big oof, considering it's a python and usually nonworking code is good enough to generate byte code. (also it has hard-coded dtype and device)
-
punctuator2
A bidirectional recurrent neural network model with attention mechanism for restoring missing punctuation in unsegmented text
-
Flux is quite a nice lower level library:
https://github.com/FluxML/Flux.jl
On top of that there are many higher level libraries such as Transformers.jl
-
open-musiclm
Implementation of MusicLM, a text to music model published by Google Research, with a few modifications.
I'm working on a personal project where I'm planning to create a fine-tuned version of Music LM, but I'm not sure if there are any pretrained weights that I can use that are available to the public. The closest thing that I can find is this: GitHub - zhvng/open-musiclm: Implementation of MusicLM, a text to music model published by Google Research, with a few modifications.which is a modified, custom version of MusicLM, but I'm looking for something closer to the paper. Does anyone have any resources on this?
-
PEN-Net-for-Inpainting
[CVPR'2019] PEN-Net: Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting
-
TokenCut
(CVPR 2022) Pytorch implementation of "Self-supervised transformers for unsupervised object discovery using normalized cut"
-
-
DeepViewAgg
[CVPR'22 Best Paper Finalist] Official PyTorch implementation of the method presented in "Learning Multi-View Aggregation In the Wild for Large-Scale 3D Semantic Segmentation"
-
gnn-lspe
Source code for GNN-LSPE (Graph Neural Networks with Learnable Structural and Positional Representations), ICLR 2022
-
ScreenAI
Implementation of the ScreenAI model from the paper: "A Vision-Language Model for UI and Infographics Understanding"
Project mention: Implementation of Google's ScreenAI: Vision-Lang Model for UI and Understanding | news.ycombinator.com | 2024-04-09 -
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
The latest post mention was on 2024-04-09.
attention related posts
- ElevenLabs Launches Voice Translation Tool to Break Down Language Barriers
- LongLlama
- LongNet: Scaling Transformers to 1,000,000,000 Tokens
- Which features you wish that were added to Character Ai?
- Why AI will not replace programmers.
- An open model that beats ChatGPT. We're seeing a real shift towards open source models that will accelerate in the coming weeks.
- GitHub - kyegomez/LongNet: Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
-
A note from our sponsor - InfluxDB
www.influxdata.com | 18 Apr 2024
Index
What are some of the best open-source attention projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | nn | 47,503 |
| 2 | nlp-tutorial | 13,666 |
| 3 | attention-is-all-you-need-pytorch | 8,409 |
| 4 | pytorch-seq2seq | 5,138 |
| 5 | scenic | 2,985 |
| 6 | pytorch-GAT | 2,222 |
| 7 | transformer-pytorch | 2,106 |
| 8 | ABSA-PyTorch | 1,942 |
| 9 | gansformer | 1,302 |
| 10 | performer-pytorch | 1,029 |
| 11 | graphtransformer | 776 |
| 12 | long-range-arena | 680 |
| 13 | LongNet | 650 |
| 14 | punctuator2 | 648 |
| 15 | Transformers.jl | 501 |
| 16 | open-musiclm | 474 |
| 17 | PEN-Net-for-Inpainting | 343 |
| 18 | TokenCut | 285 |
| 19 | how_attentive_are_gats | 275 |
| 20 | DeepViewAgg | 216 |
| 21 | gnn-lspe | 214 |
| 22 | ScreenAI | 147 |
| 23 | Perceiver | 85 |
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com