Reweight-gpt Alternatives
Similar projects and alternatives to reweight-gpt
-
-
CodeRabbit
CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
-
ai_story_scale
The AI story scale (AISS): A human rating scale for texts written with generative language models.
-
ML-foundations
Machine Learning Foundations: Linear Algebra, Calculus, Statistics & Computer Science
-
-
-
numerical-linear-algebra
Free online textbook of Jupyter notebooks for fast.ai Computational Linear Algebra course
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
reweight-gpt discussion
reweight-gpt reviews and mentions
-
[Research] An alternative to self-attention mechanism in GPT
Instead of self attention, I tried to generate the self-attention matrix directly using lateral connections among the inputs. The method is like LSTM but it gates all the past inputs using separate gates for each input (it can be parallelized). It's very easy to implement the method into the current GPT architectures. You just remove the attention part and replace it with learnable weights. Her is a working implementation (around100 lines!): Code: https://github.com/hunar4321/reweight-gpt In my experience, it learns very well and it can super-pass the self-attention mechanism if the number of the parameters are matched. (I tested it on small datasets for next character prediction. I haven't systematically compared these two methods yet).
Stats
hunar4321/reweight-gpt is an open source project licensed under MIT License which is an OSI approved license.
The primary programming language of reweight-gpt is Jupyter Notebook.
Popular Comparisons
Sponsored