How to use AMD GPU?

• text-generation-webui

  876 36,293 9.9 Python

  A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

git clone https://github.com/oobabooga/text-generation-webui.git cd text-generation-webui

• bitsandbytes-rocm

  2 36 3.6 Python

  8-bit CUDA functions for PyTorch, ported to HIP for use in AMD GPUs (by agrocylo)

pip uninstall bitsandbytes cd .. git clone https://github.com/agrocylo/bitsandbytes-rocm.git cd bitsandbytes-rocm make hip python setup.py install

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• triton

  2 70 9.7 C++

  Development repository for the Triton language and compiler (by ROCm)

  

cd .. git clone https://github.com/ROCmSoftwarePlatform/triton.git -b release/pytorch_2.0 cd triton/python pip3 install cmake pip3 install -e .

• GPTQ-for-LLaMa

  75 2,913 8.6 Python

  4 bits quantization of LLaMA using GPTQ

cd ../.. git clone https://github.com/qwopqwop200/GPTQ-for-LLaMa.git -b triton cd GPTQ-for-LLaMa pip install -r requirements.txt mkdir -p ../text-generation-webui/repositories ln -s ../../GPTQ-for-LLaMa ../text-generation-webui/repositories/GPTQ-for-LLaMa

Suggest a related project