Basic-UI-for-GPT-J-6B-with-low-vram
A repository to run gpt-j-6b on low vram machines (4.2 gb minimum vram for 2000 token context, 3.5 gb for 1000 token context). Model loading takes 12gb free ram. (by arrmansa)
nn
🧑🏫 60 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠 (by lab-ml)
| Basic-UI-for-GPT-J-6B-with-low-vram | nn | |
|---|---|---|
| 4 | 26 | |
| 113 | 48,430 | |
| - | 4.5% | |
| 0.0 | 7.7 | |
| over 2 years ago | about 1 month ago | |
| Jupyter Notebook | Jupyter Notebook | |
| Apache License 2.0 | MIT License |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Basic-UI-for-GPT-J-6B-with-low-vram
Posts with mentions or reviews of Basic-UI-for-GPT-J-6B-with-low-vram.
We have used some of these posts to build our list of alternatives
and similar projects.
-
How to run this service with a local GPU?
You need a lot of VRAM to run the AI models, scaling somewhat with the amount of parameters a model uses. The most advanced model Pygmalion has is 6 billion parameters, which requires a minimum of 16GB of VRAM to run locally at decent speeds. There are methods of running 6b locally on low VRAM machines as listed here: https://github.com/arrmansa/Basic-UI-for-GPT-J-6B-with-low-vram but even then, the generations would be excruciatingly slow, and the lowest VRAM card used with this method has 6GB of VRAM.
-
Tesla M40 and GPT-J-6B
While waiting however I came across https://github.com/arrmansa/Basic-UI-for-GPT-J-6B-with-low-vram which allows you to use some of system memory to run the model. I was able to get a version working with 2.7B on my 2060 6GB with KoboldAI. The github above has an error that prevents it from working (https://github.com/arrmansa/Basic-UI-for-GPT-J-6B-with-low-vram/issues/1), but other than that it works.
-
How is any of this even possible?
Just to add to this, there is a low VRAM version of GPT-J here (suggest 16GB RAM + 8GB GPU).
-
GPT-J 6B locally on my computer
I found this yesterday, is it somehow possible to use this with KoboldAI to run GPT-J on weaker graphics cards?
nn
Posts with mentions or reviews of nn.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-01-09.
-
Can't remember name of website that has explanations side-by-side with code
Hey are you talking about https://nn.labml.ai/ ?
- [D] Recent ML papers to implement from scratch
-
[P] GPT-NeoX inference with LLM.int8() on 24GB GPU
Implementation & LM Eval Harness Results
-
[P] Fine-tuned the GPT-Neox Model to Generate Quotes
Github: https://github.com/labmlai/annotated_deep_learning_paper_implementations/tree/master/labml_nn/neox
-
Best resources to learn recent transformer papers and stay updated [D]
Regarding implementations this helps me: https://nn.labml.ai/
- Introductory papers to implement
- How to convert research papers to code?
-
[D] How to convert papers to code?
Dunno if this is directly helpful, but this website has implementation with the math side by side https://nn.labml.ai/
- [D] Looking for open source projects to contribute
- Resource for papers explanation
What are some alternatives?
When comparing Basic-UI-for-GPT-J-6B-with-low-vram and nn you can also consider the following projects:
gpt-neo_dungeon - Colab notebooks to run a basic AI Dungeon clone using gpt-neo-2.7B
GFPGAN-for-Video-SR - A colab notebook for video super resolution using GFPGAN
adaptnlp - An easy to use Natural Language Processing library and framework for predicting, training, fine-tuning, and serving up state-of-the-art NLP models.
labml - 🔎 Monitor deep learning model training and hardware usage from your mobile phone 📱
Behavior-Sequence-Transformer-Pytorch - This is a pytorch implementation for the BST model from Alibaba https://arxiv.org/pdf/1905.06874.pdf
functorch - functorch is JAX-like composable function transforms for PyTorch.
clip-italian - CLIP (Contrastive Language–Image Pre-training) for Italian
ZoeDepth - Metric depth estimation from a single image
pytorch-sentiment-analysis - Tutorials on getting started with PyTorch and TorchText for sentiment analysis.
onnx-simplifier - Simplify your onnx model
pytorch-generative - Easy generative modeling in PyTorch.
Basic-UI-for-GPT-J-6B-with-low-vram vs gpt-neo_dungeon
nn vs GFPGAN-for-Video-SR
Basic-UI-for-GPT-J-6B-with-low-vram vs adaptnlp
nn vs labml
Basic-UI-for-GPT-J-6B-with-low-vram vs Behavior-Sequence-Transformer-Pytorch
nn vs functorch
Basic-UI-for-GPT-J-6B-with-low-vram vs clip-italian
nn vs ZoeDepth
Basic-UI-for-GPT-J-6B-with-low-vram vs pytorch-sentiment-analysis
nn vs onnx-simplifier
Basic-UI-for-GPT-J-6B-with-low-vram vs pytorch-generative
nn vs Behavior-Sequence-Transformer-Pytorch