The installed version of bitsandbytes was compiled without GPU support

• text-generation-webui

  876 36,293 9.9 Python

  A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

# remover everything sudo apt-get remove --purge '*nvidia*' # download and installed last nvidia drivers from settings -> driver manager -> software sources: NVIDIA driver metapackage from nvidia-driver-525 (proprietary, tested) # install GUI # from https://github.com/oobabooga/text-generation-webui conda activate conda create -n chat python=3.10.9 conda activate chat pip3 install torch torchvision torchaudio git clone https://github.com/oobabooga/text-generation-webui cd text-generation-webui pip install -r requirements.txt pip install -r requirements.txt --upgrade

• GPTQ-for-LLaMa

  75 2,913 8.6 Python

  4 bits quantization of LLaMA using GPTQ

# To use the GPTQ models I need to Install GPTQ-for-LLaMa and the monkey patch mkdir repositories cd repositories git clone https://github.com/qwopqwop200/GPTQ-for-LLaMa.git -b triton cd GPTQ-for-LLaMa pip install ninja pip install -r requirements.txt cd cd text-generation-webui # download random model python download-model.py xxx/yyy # try to start the gui python server.py # It returns this warning but it runs bin /home/gm/miniconda3/envs/chat/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so /home/gm/miniconda3/envs/chat/lib/python3.10/site-packages/bitsandbytes/cextension.py:34: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable. warn("The installed version of bitsandbytes was compiled without GPU support. " /home/gm/miniconda3/envs/chat/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so: undefined symbol: cadam32bit_grad_fp32

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project