Top 5 Python attention-is-all-you-need Projects

attention-is-all-you-need-pytorch

3 8,432 0.0 Python

A PyTorch implementation of the Transformer model in "Attention is All You Need".

Project mention: ElevenLabs Launches Voice Translation Tool to Break Down Language Barriers | news.ycombinator.com | 2023-10-10

The transformer model was invented to attend to context over the entire sequence length. Look at how the original authors used the Transformer for NMT in the original Vaswani et al publication. https://github.com/jadore801120/attention-is-all-you-need-py...

whisper-timestamped

2 1,501 8.3 Python

Multilingual Automatic Speech Recognition with word-level timestamps and confidence

Project mention: Show HN: AI Dub Tool I Made to Watch Foreign Language Videos with My 7-Year-Old | news.ycombinator.com | 2024-02-28

Yes. But Whisper's word-level timings are actually quite inaccurate out of the box. There are some Python libraries that mitigate that. I tested several of them. whisper-timestamped seems to be the best one. [0]
[0] https://github.com/linto-ai/whisper-timestamped

WorkOS

workos.com sponsored

The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
LongNet

16 650 9.0 Python

Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"

Project mention: LongLlama | /r/LocalLLaMA | 2023-07-07

If you want to talk immature looking, longnet wouldn't even compile. That's a big oof, considering it's a python and usually nonworking code is good enough to generate byte code. (also it has hard-coded dtype and device)

ScreenAI

1 174 7.1 Python

Implementation of the ScreenAI model from the paper: "A Vision-Language Model for UI and Infographics Understanding"

Project mention: Implementation of Google's ScreenAI: Vision-Lang Model for UI and Understanding | news.ycombinator.com | 2024-04-09

a-PyTorch-Tutorial-to-Transformers

1 159 5.3 Python

Attention Is All You Need | a PyTorch Tutorial to Transformers

Project mention: Transformers Tutorial - learn to implement transformers from scratch | /r/learnmachinelearning | 2023-05-01

InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).