Reweight-gpt Alternatives
Similar projects and alternatives to reweight-gpt
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
-
ai_story_scale
Discontinued The AI story scale (AISS): A human rating scale for texts written with generative language models. [GET https://api.github.com/repos/MWiechmann/ai_story_scale: 404 - Not Found // See: https://docs.github.com/rest/repos/repos#get-a-repository]
-
ML-foundations
Machine Learning Foundations: Linear Algebra, Calculus, Statistics & Computer Science
-
-
numerical-linear-algebra
Free online textbook of Jupyter notebooks for fast.ai Computational Linear Algebra course
-
-
machine_learning_complete
A comprehensive machine learning repository containing 30+ notebooks on different concepts, algorithms and techniques.
reweight-gpt discussion
reweight-gpt reviews and mentions
-
[Research] An alternative to self-attention mechanism in GPT
Instead of self attention, I tried to generate the self-attention matrix directly using lateral connections among the inputs. The method is like LSTM but it gates all the past inputs using separate gates for each input (it can be parallelized). It's very easy to implement the method into the current GPT architectures. You just remove the attention part and replace it with learnable weights. Her is a working implementation (around100 lines!): Code: https://github.com/hunar4321/reweight-gpt In my experience, it learns very well and it can super-pass the self-attention mechanism if the number of the parameters are matched. (I tested it on small datasets for next character prediction. I haven't systematically compared these two methods yet).
Stats
hunar4321/reweight-gpt is an open source project licensed under MIT License which is an OSI approved license.
The primary programming language of reweight-gpt is Jupyter Notebook.
Popular Comparisons
Sponsored