Error on model load: Torch not compiled with CUDA enabled

• bitsandbytes-windows-webui

  4 333 8.1 HTML

  Windows compile of bitsandbytes for use in text-generation-webui.

set-executionpolicy RemoteSigned -Scope CurrentUser python -m venv venv venv\Scripts\Activate.ps1 pip install torchaudio torch==2.0.0+cu118 torchvision ninja --force-reinstall --extra-index-url https://download.pytorch.org/whl/cu118 pip install https://github.com/jllllll/bitsandbytes-windows-webui/raw/main/bitsandbytes-0.38.1-py3-none-any.whl pip install -r requirements.txt mkdir repositories cd repositories git clone https://github.com/oobabooga/GPTQ-for-LLaMa.git -b cuda cd GPTQ-for-LLaMa python -m pip install -r requirements.txt python setup_cuda.py install python server.py --chat --model-menu

• GPTQ-for-LLaMa

  19 129 7.7 Python

  4 bits quantization of LLaMa using GPTQ (by oobabooga)

set-executionpolicy RemoteSigned -Scope CurrentUser python -m venv venv venv\Scripts\Activate.ps1 pip install torchaudio torch==2.0.0+cu118 torchvision ninja --force-reinstall --extra-index-url https://download.pytorch.org/whl/cu118 pip install https://github.com/jllllll/bitsandbytes-windows-webui/raw/main/bitsandbytes-0.38.1-py3-none-any.whl pip install -r requirements.txt mkdir repositories cd repositories git clone https://github.com/oobabooga/GPTQ-for-LLaMa.git -b cuda cd GPTQ-for-LLaMa python -m pip install -r requirements.txt python setup_cuda.py install python server.py --chat --model-menu

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project