Transformer-in-Transformer
LongNet
Our great sponsors
| Transformer-in-Transformer | LongNet | |
|---|---|---|
| 4 | 16 | |
| 41 | 651 | |
| - | - | |
| 0.0 | 9.0 | |
| about 2 years ago | 4 months ago | |
| Jupyter Notebook | Python | |
| Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Transformer-in-Transformer
- I Implemented Transformer in Transformer
-
Hacker News top posts: Dec 6, 2021
I Implemented Transformer in Transformer\ (5 comments)
- [P] I implemented Transformer in Transformer
LongNet
-
Which features you wish that were added to Character Ai?
i wish they would implement this into character.ai github.com/kyegomez/LongNet
- Why AI will not replace programmers.
-
LongLlama
If you want to talk immature looking, longnet wouldn't even compile. That's a big oof, considering it's a python and usually nonworking code is good enough to generate byte code. (also it has hard-coded dtype and device)
-
An open model that beats ChatGPT. We're seeing a real shift towards open source models that will accelerate in the coming weeks.
When will the Open Source LLMs start using LongNet https://github.com/kyegomez/LongNet https://arxiv.org/abs/2307.02486
- GitHub - kyegomez/LongNet: Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
What are some alternatives?
poolformer - PoolFormer: MetaFormer Is Actually What You Need for Vision (CVPR 2022 Oral)
long_llama - LongLLaMA is a large language model capable of handling long contexts. It is based on OpenLLaMA and fine-tuned with the Focused Transformer (FoT) method.
AvatarGAN - Generate Cartoon Images using Generative Adversarial Network
unilm - Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
swarms - Orchestrate Swarms of Agents From Any Framework Like OpenAI, Langchain, and Etc for Real World Workflow Automation. Join our Community: https://discord.gg/DbjBMJTSWD
PromptBroker - 🦊 The ONLY AI Prompts Broker you will ever need.
principia - The Principia Rewrite
a-PyTorch-Tutorial-to-Transformers - Attention Is All You Need | a PyTorch Tutorial to Transformers
planckforth - Bootstrapping a Forth interpreter from hand-written tiny ELF binary. Just for fun.
Fast-Transformer - An implementation of Fastformer: Additive Attention Can Be All You Need, a Transformer Variant in TensorFlow
Play-Billing-v6-For-Unity - A Plugin for Unity which implements Google Play Billing Library v6.0.1 for in app products, made (mostly) by ChatGPT and GPT-4.