bitsandbytes-rocm
By broncotc
GPTQ-for-LLaMa
4 bits quantization of LLMs using GPTQ (by YellowRoseCx)
| bitsandbytes-rocm | GPTQ-for-LLaMa | |
|---|---|---|
| 4 | 1 | |
| 38 | 0 | |
| - | - | |
| 8.8 | 8.2 | |
| about 1 year ago | 11 months ago | |
| Python | Python | |
| MIT License | Apache License 2.0 |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
bitsandbytes-rocm
Posts with mentions or reviews of bitsandbytes-rocm.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-06-22.
-
Any methods to train using AMD?
Install dependencies like hipblas-devel hipsparse-devel hipcub-devel git python3.10 make libstdc++-devel accelerate, rocm and hip git clone https://github.com/bmaltais/kohya_ss && cd kohya_ss python3.10 -m venv venv source venv/bin/activate pip3 install torch==1.13.1 torchvision==0.14.1 torchtext==0.14.1 torchaudio==0.13.1 --index-url https://download.pytorch.org/whl/rocm5.2 # problems on 2.0.0 last I tried, but kohya gotten updates since then pip3 install --upgrade -r requirements.txt pip3 uninstall tensorflow && pip3 install tensorflow-rocm pip uninstall bitsandbytes && git clone https://github.com/broncotc/bitsandbytes-rocm # bitsandbytes not required if not using adam8? cd bitesandbytes-rocm && nano Makefile # Replace all 3 instances of 5.3.0 with 5.4.3 make hip python3 setup.py install
-
How to run Pygmalion on 4.5GB of VRAM with full context size.
There are a lot of ROCm versions of bitsandbytes. For example this one: https://github.com/broncotc/bitsandbytes-rocm The problem is compatibility with most of the requirements. Kobold does a better job than ooba in offering a more streamlined approach for AMD users.
- Is it possible to load a model in 8bit precision with an AMD card? (6700xt)
- Have you got running LoRA training on an AMD GPU?
GPTQ-for-LLaMa
Posts with mentions or reviews of GPTQ-for-LLaMa.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-04-02.
-
How to run Pygmalion on 4.5GB of VRAM with full context size.
I can load it with GPTQ on a 6700xt using a fork: https://github.com/YellowRoseCx/GPTQ-for-LLaMa You can update the post for AMD users if you want.
What are some alternatives?
When comparing bitsandbytes-rocm and GPTQ-for-LLaMa you can also consider the following projects:
sd-scripts
bitsandbytes - Accessible large language models via k-bit quantization for PyTorch.
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
LoRA_Easy_Training_Scripts - A UI made in Pyside6 to make training LoRA/LoCon and other LoRA type models in sd-scripts easy
LyCORIS - Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion.