Python self-attention

Open-source Python projects categorized as self-attention
Edit details
Topics: Pytorch Transformer attention Deep Learning time-series
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
Sponsored

Top 14 Python self-attention Projects

  • Informer2020

    The GitHub repository for the paper "Informer" accepted by AAAI 2021.

  • GAT

    Graph Attention Networks (https://arxiv.org/abs/1710.10903)

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • DeBERTa

    The implementation of DeBERTa

  • how-do-vits-work

    (ICLR 2022 Spotlight) Official PyTorch implementation of "How Do Vision Transformers Work?"

  • GCVit

    [ICML 2023] Official PyTorch implementation of Global Context Vision Transformers

  • SAITS

    The official PyTorch implementation of the paper "SAITS: Self-Attention-based Imputation for Time Series". A fast and state-of-the-art (SOTA) deep-learning neural network model for efficient time-series imputation (impute multivariate incomplete time series containing NaN missing data/values with machine learning). https://arxiv.org/abs/2202.08516

    • Project mention: SAITS: Self-Attention-based Imputation for Time Series. Expert Systems with Applications, 219:119619, 2023. | /r/science | 2023-07-13

    The full paper is available on arXiv: https://arxiv.org/abs/2202.08516. The code on GitHub: https://github.com/WenjieDu/SAITS/. If your research lies in time-series modeling, you may also be interested in the work PyPOTS: a Python toolbox for data mining on Partially-Observed Time Series. Its full paper is available on arXiv as well https://arxiv.org/abs/2305.18811, which has been peer-reviewed and accepted by the 9th SIGKDD international workshop Mining and Learning from Time Series (MiLeTS'23).

  • Parallel-Tacotron2

    PyTorch Implementation of Google's Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modeling

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo

  • ViTGAN

    A PyTorch implementation of ViTGAN based on paper ViTGAN: Training GANs with Vision Transformers.

  • ubisoft-laforge-daft-exprt

    PyTorch Implementation of Daft-Exprt: Robust Prosody Transfer Across Speakers for Expressive Speech Synthesis

  • SLiMe

    1-shot image segmentation using Stable Diffusion (by aliasgharkhani)

    • Project mention: SLiMe: One-shot image segmentation using Stable Diffusion | news.ycombinator.com | 2023-10-18

  • CrabNet

    Predict materials properties using only the composition information!

    • Project mention: Artificial intelligence can revolutionise science | news.ycombinator.com | 2023-09-15

    I don't know. As for "literature-based discovery," this project/paper sounded like a pretty big deal when it came out a few years ago: https://github.com/materialsintelligence/mat2vec . And I see this thing came out more recently: https://github.com/anthony-wang/CrabNet .

    Of course not all fields lend themselves as well to this as does materials science.

  • query-selector

    LONG-TERM SERIES FORECASTING WITH QUERYSELECTOR – EFFICIENT MODEL OF SPARSEATTENTION

  • AREnets

    Tensorflow-based framework which lists attentive implementation of the conventional neural network models (CNN, RNN-based), applicable for Relation Extraction classification tasks as well as API for custom model implementation

  • attention_to_gif

    Visualize transition of attention weights across layers in a Transformer as a GIF

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Python self-attention related posts

Index

What are some of the best open-source self-attention projects in Python? This list will help you:

Project Stars
1 Informer2020 4,890
2 GAT 3,045
3 DeBERTa 1,844
4 how-do-vits-work 784
5 GCVit 414
6 SAITS 252
7 Parallel-Tacotron2 184
8 ViTGAN 155
9 ubisoft-laforge-daft-exprt 114
10 SLiMe 106
11 CrabNet 81
12 query-selector 74
13 AREnets 7
14 attention_to_gif 1

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com