reweight-gpt
Reweight GPT - a simple neural network using transformer architecture for next character prediction (by hunar4321)
Andrew-NG-Notes
This is Andrew NG Coursera Handwritten Notes. (by ashishpatel26)
| reweight-gpt | Andrew-NG-Notes | |
|---|---|---|
| 1 | 1 | |
| 51 | 2,851 | |
| - | 0.6% | |
| 6.3 | 0.0 | |
| over 1 year ago | about 1 year ago | |
| Jupyter Notebook | Jupyter Notebook | |
| MIT License | - |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
reweight-gpt
Posts with mentions or reviews of reweight-gpt.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-05-01.
-
[Research] An alternative to self-attention mechanism in GPT
Instead of self attention, I tried to generate the self-attention matrix directly using lateral connections among the inputs. The method is like LSTM but it gates all the past inputs using separate gates for each input (it can be parallelized). It's very easy to implement the method into the current GPT architectures. You just remove the attention part and replace it with learnable weights. Her is a working implementation (around100 lines!): Code: https://github.com/hunar4321/reweight-gpt In my experience, it learns very well and it can super-pass the self-attention mechanism if the number of the parameters are matched. (I tested it on small datasets for next character prediction. I haven't systematically compared these two methods yet).
Andrew-NG-Notes
Posts with mentions or reviews of Andrew-NG-Notes.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-04-18.
What are some alternatives?
When comparing reweight-gpt and Andrew-NG-Notes you can also consider the following projects:
repeng - A library for making RepE control vectors
Note - Machine learning library, Distributed training, Deep learning, Reinforcement learning, Models, TensorFlow, PyTorch
ai_story_scale - The AI story scale (AISS): A human rating scale for texts written with generative language models.
DeepNeuralNetworksFromScratch - Different kinds of deep neural networks (DNNs) implemented from scratch using Python and NumPy, with a TensorFlow-like object-oriented API.
ML-experiments
machine_learning_complete - A comprehensive machine learning repository containing 30+ notebooks on different concepts, algorithms and techniques.