awesome-fast-attention
LongNet
| awesome-fast-attention | LongNet | |
|---|---|---|
| 1 | 16 | |
| 827 | 652 | |
| - | - | |
| 1.0 | 9.0 | |
| over 2 years ago | 4 months ago | |
| Python | Python | |
| GNU General Public License v3.0 only | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
awesome-fast-attention
LongNet
-
Which features you wish that were added to Character Ai?
i wish they would implement this into character.ai github.com/kyegomez/LongNet
- Why AI will not replace programmers.
-
LongLlama
If you want to talk immature looking, longnet wouldn't even compile. That's a big oof, considering it's a python and usually nonworking code is good enough to generate byte code. (also it has hard-coded dtype and device)
-
An open model that beats ChatGPT. We're seeing a real shift towards open source models that will accelerate in the coming weeks.
When will the Open Source LLMs start using LongNet https://github.com/kyegomez/LongNet https://arxiv.org/abs/2307.02486
- GitHub - kyegomez/LongNet: Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
What are some alternatives?
how-do-vits-work - (ICLR 2022 Spotlight) Official PyTorch implementation of "How Do Vision Transformers Work?"
Transformer-in-Transformer - An Implementation of Transformer in Transformer in TensorFlow for image classification, attention inside local patches
a-PyTorch-Tutorial-to-Transformers - Attention Is All You Need | a PyTorch Tutorial to Transformers
long_llama - LongLLaMA is a large language model capable of handling long contexts. It is based on OpenLLaMA and fine-tuned with the Focused Transformer (FoT) method.
unilm - Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
PromptBroker - 🦊 The ONLY AI Prompts Broker you will ever need.
swarms - Orchestrate Swarms of Agents From Any Framework Like OpenAI, Langchain, and Etc for Real World Workflow Automation. Join our Community: https://discord.gg/DbjBMJTSWD
Play-Billing-v6-For-Unity - A Plugin for Unity which implements Google Play Billing Library v6.0.1 for in app products, made (mostly) by ChatGPT and GPT-4.
nn - 🧑🏫 60 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠