LaMDA-rlhf-pytorch
Perceiver
| LaMDA-rlhf-pytorch | Perceiver | |
|---|---|---|
| 7 | 7 | |
| 454 | 85 | |
| - | - | |
| 3.0 | 2.6 | |
| 3 months ago | about 3 years ago | |
| Python | Python | |
| MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
LaMDA-rlhf-pytorch
-
All subscribers should have the same level of access
I made a mistake about the LaMDa, what I mean is the machine learning model that Lamda based on: Transformer. Though you can still access to the paper: [2201.08239] LaMDA: Language Models for Dialog Applications (arxiv.org) and a pyTorch implementation here: conceptofmind/LaMDA-rlhf-pytorch: Open-source pre-training implementation of Google's LaMDA in PyTorch. Adding RLHF similar to ChatGPT. (github.com)
-
Closest Open Source alternative to CharacterAI?
I know it'll be 10 times worse for now at best since this is refined proprietary tech where a lot of resources went into but I'm still looking forward to something aiming to achieve this level of fluency and context understanding that can be used open and freely. Closest thing I could find so far might be GPT2 or laMDA?
- LaMDA PyTorch
-
Open-Source LaMDA Model
An open-source implementation for the pre-training architecture of Google's LaMDA in PyTorch. The research paper outlines an autoregressive, decoder-only, GPT-like transformer language model. The transformer uses T5 relative positional bias in the attention layers and gated-GELU activation function in the feed-forward layers.
The repository currently contains a script for basic training as well as Huggingface datasets and Weights & Biases integration.
LaMDA research paper: https://arxiv.org/abs/2201.08239
Github repository for the model: https://github.com/conceptofmind/LaMDA-pytorch
The pre-training architecture was peer-reviewed by Dr. Phil Wang. Please check out and support his work: https://github.com/lucidrains.
Updates: https://twitter.com/EnricoShippole
-
[D] What if sentient AI has already taken over without us knowing?
If you want to view the pre-training architecture of a transformer model such as LaMDA, the one you are referencing in this post, you can view that here: https://github.com/conceptofmind/LaMDA-pytorch/blob/main/lamda_pytorch/lamda_pytorch.py
-
[D] length of input sequence for transformers?
An example for an iterable data loader like this would be: https://github.com/conceptofmind/LaMDA-pytorch/blob/main/dataloader/stream_dataloader.py
-
[P] Open-source LaMDA Model
Github repository for the model: https://github.com/conceptofmind/LaMDA-pytorch
Perceiver
- I implemented Deepmind's new Perceiver Model
- I Implemented Deepmind's Perceiver Model
-
[P] I implemented DeepMind's "Perceiver" in PyTorch
Great one, I implemented the Perceiver model too in TensorFlow: https://github.com/Rishit-dagli/Perceiver
- Deepmind's New Perceiver Model
-
[P] Implementing Perceiver: General perception with Iterative Attention in TensorFlow
The project: https://github.com/Rishit-dagli/Perceiver
- Perceiver, General Perception with Iterative Attention
What are some alternatives?
lamda-pytorch - Open-source pre-training implementation of Google's LaMDA in PyTorch. Adding RLHF similar to ChatGPT. [Moved to: https://github.com/conceptofmind/LaMDA-rlhf-pytorch]
Swin-Transformer-Object-Detection - This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" on Object Detection and Instance Segmentation.
Conformer - An implementation of Conformer: Convolution-augmented Transformer for Speech Recognition, a Transformer Variant in TensorFlow/Keras
performer-pytorch - An implementation of Performer, a linear attention-based transformer, in Pytorch
awesome-RLHF - A curated list of reinforcement learning with human feedback resources (continually updated)
Fast-Transformer - An implementation of Fastformer: Additive Attention Can Be All You Need, a Transformer Variant in TensorFlow
iris - Transformers are Sample-Efficient World Models. ICLR 2023, notable top 5%.
TimeSformer-pytorch - Implementation of TimeSformer from Facebook AI, a pure attention-based solution for video classification
PaLM-rlhf-pytorch - Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM
gato - Unofficial Gato: A Generalist Agent
deepmind-perceiver - My implementation of DeepMind's Perceiver
conformer - Implementation of the convolutional module from the Conformer paper, for use in Transformers