reweight-gpt
Reweight GPT - a simple neural network using transformer architecture for next character prediction (by hunar4321)
ai_story_scale
The AI story scale (AISS): A human rating scale for texts written with generative language models. (by MWiechmann)
| reweight-gpt | ai_story_scale | |
|---|---|---|
| 1 | 2 | |
| 51 | 9 | |
| - | - | |
| 6.3 | 3.6 | |
| over 1 year ago | 10 months ago | |
| Jupyter Notebook | Jupyter Notebook | |
| MIT License | Creative Commons Attribution Share Alike 4.0 |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
reweight-gpt
Posts with mentions or reviews of reweight-gpt.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-05-01.
-
[Research] An alternative to self-attention mechanism in GPT
Instead of self attention, I tried to generate the self-attention matrix directly using lateral connections among the inputs. The method is like LSTM but it gates all the past inputs using separate gates for each input (it can be parallelized). It's very easy to implement the method into the current GPT architectures. You just remove the attention part and replace it with learnable weights. Her is a working implementation (around100 lines!): Code: https://github.com/hunar4321/reweight-gpt In my experience, it learns very well and it can super-pass the self-attention mechanism if the number of the parameters are matched. (I tested it on small datasets for next character prediction. I haven't systematically compared these two methods yet).
ai_story_scale
Posts with mentions or reviews of ai_story_scale.
We have used some of these posts to build our list of alternatives
and similar projects.
-
Survey Results Preview (90% Progress Milestone): Morpho
To get a better idea of what is going on with Morpho stories, here are two typical outputs from the Morpho preset.
-
Survey Results Preview III (75% Progress): Basic Coherence
Two typical outputs are here.
What are some alternatives?
When comparing reweight-gpt and ai_story_scale you can also consider the following projects:
repeng - A library for making RepE control vectors
augmented-interpretable-models - Interpretable and efficient predictors using pre-trained language models. Scikit-learn compatible.
ML-experiments
Smarty-GPT - A wrapper of LLMs that biases its behaviour using prompts and contexts in a transparent manner to the end-users
Andrew-NG-Notes - This is Andrew NG Coursera Handwritten Notes.
language-planner - Official Code for "Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents"