Top 23 Python attention Projects

• attention-is-all-you-need-pytorch

  3 8,432 0.0 Python

  A PyTorch implementation of the Transformer model in "Attention is All You Need".

Project mention: ElevenLabs Launches Voice Translation Tool to Break Down Language Barriers | news.ycombinator.com | 2023-10-10

The transformer model was invented to attend to context over the entire sequence length. Look at how the original authors used the Transformer for NMT in the original Vaswani et al publication. https://github.com/jadore801120/attention-is-all-you-need-py...

• scenic

  5 2,995 8.6 Python

  Scenic: A Jax Library for Computer Vision Research and Beyond (by google-research)

  

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• transformer-pytorch

  2 2,152 2.1 Python

  Transformer: PyTorch Implementation of "Attention Is All You Need"

• ABSA-PyTorch

  1 1,945 0.0 Python

  Aspect Based Sentiment Analysis, PyTorch Implementations. 基于方面的情感分析，使用PyTorch实现。

• gansformer

  7 1,302 1.8 Python

  Generative Adversarial Transformers

• performer-pytorch

  2 1,029 1.8 Python

  An implementation of Performer, a linear attention-based transformer, in Pytorch

• graphtransformer

  4 804 0.0 Python

  Graph Transformer Architecture. Source code for "A Generalization of Transformer Networks to Graphs", DLG-AAAI'21.

  

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• long-range-arena

  6 682 0.0 Python

  Long Range Arena for Benchmarking Efficient Transformers

  

Project mention: The Secret Sauce behind 100K context window in LLMs: all tricks in one place | news.ycombinator.com | 2023-06-17

https://github.com/google-research/long-range-arena

• LongNet

  16 650 9.0 Python

  Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"

Project mention: LongLlama | /r/LocalLLaMA | 2023-07-07

If you want to talk immature looking, longnet wouldn't even compile. That's a big oof, considering it's a python and usually nonworking code is good enough to generate byte code. (also it has hard-coded dtype and device)

• punctuator2

  3 648 1.8 Python

  A bidirectional recurrent neural network model with attention mechanism for restoring missing punctuation in unsegmented text

• open-musiclm

  1 477 7.0 Python

  Implementation of MusicLM, a text to music model published by Google Research, with a few modifications.

Project mention: [Research] Pretrained MusicLM? | /r/MachineLearning | 2023-09-03

I'm working on a personal project where I'm planning to create a fine-tuned version of Music LM, but I'm not sure if there are any pretrained weights that I can use that are available to the public. The closest thing that I can find is this: GitHub - zhvng/open-musiclm: Implementation of MusicLM, a text to music model published by Google Research, with a few modifications.which is a modified, custom version of MusicLM, but I'm looking for something closer to the paper. Does anyone have any resources on this?

• PEN-Net-for-Inpainting

  1 343 0.0 Python

  [CVPR'2019] PEN-Net: Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting

  

• how_attentive_are_gats

  1 275 0.0 Python

  Code for the paper "How Attentive are Graph Attention Networks?" (ICLR'2022)

  

• DeepViewAgg

  4 215 4.8 Python

  [CVPR'22 Best Paper Finalist] Official PyTorch implementation of the method presented in "Learning Multi-View Aggregation In the Wild for Large-Scale 3D Semantic Segmentation"

• gnn-lspe

  3 214 0.0 Python

  Source code for GNN-LSPE (Graph Neural Networks with Learnable Structural and Positional Representations), ICLR 2022

• ScreenAI

  1 174 7.1 Python

  Implementation of the ScreenAI model from the paper: "A Vision-Language Model for UI and Infographics Understanding"

Project mention: Implementation of Google's ScreenAI: Vision-Lang Model for UI and Understanding | news.ycombinator.com | 2024-04-09

• Perceiver

  7 85 2.6 Python

  Implementation of Perceiver, General Perception with Iterative Attention in TensorFlow

• CrabNet

  1 81 3.7 Python

  Predict materials properties using only the composition information!

Project mention: Artificial intelligence can revolutionise science | news.ycombinator.com | 2023-09-15

I don't know. As for "literature-based discovery," this project/paper sounded like a pretty big deal when it came out a few years ago: https://github.com/materialsintelligence/mat2vec . And I see this thing came out more recently: https://github.com/anthony-wang/CrabNet .

Of course not all fields lend themselves as well to this as does materials science.

• pose-transfer

  1 55 3.9 Python

  :fire: [PR 2023] Multi-scale Attention Guided Pose Transfer (official code).

Project mention: Simple pose transfer? | /r/StableDiffusion | 2023-12-05

Pose transfer should be relatively easy task nowadays but I can't find working code apart from https://github.com/prasunroy/pose-transfer which doesn't give good result (the output's face always look like a fashion model).

• LFattNet

  1 53 0.0 Python

  Attention-based View Selection Networks for Light-field Disparity Estimation

• GanVinci

  6 50 10.0 Python

  Photorealistic human image editing with GANs - Reimplementation of the paper "FEAT: Face Editing with Attention" with additional changes and improvements.

• multiscale-adversarial-attention-gates

  1 41 0.0 Python

  Code for the paper: Valvano G. et al. (2021), Learning to Segment from Scribbles using Multi-scale Adversarial Attention Gates

• vision_transformers

  1 28 7.0 Python

  Vision Transformers for image classification, image segmentation, and object detection.

• SaaSHub

  www.saashub.com sponsored

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Python attention related posts

• ElevenLabs Launches Voice Translation Tool to Break Down Language Barriers
  2 projects | news.ycombinator.com | 10 Oct 2023
• LongLlama
  2 projects | /r/LocalLLaMA | 7 Jul 2023
• LongNet: Scaling Transformers to 1,000,000,000 Tokens
  3 projects | /r/LocalLLaMA | 6 Jul 2023
• Which features you wish that were added to Character Ai?
  1 project | /r/CharacterAI | 7 Jul 2023
• Why AI will not replace programmers.
  2 projects | /r/ChatGPT | 7 Jul 2023
• An open model that beats ChatGPT. We're seeing a real shift towards open source models that will accelerate in the coming weeks.
  1 project | /r/aipromptprogramming | 6 Jul 2023
• GitHub - kyegomez/LongNet: Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
  1 project | /r/AITechTips | 6 Jul 2023
• A note from our sponsor - WorkOS
  workos.com | 25 Apr 2024

  The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning. Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com

Do not miss the trending Python projects with our weekly report!