I have tried various different methods to install, and none work. Can you spoon-feed me how?

• text-generation-webui

  876 36,293 9.9 Python

  A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

git clone https://github.com/oobabooga/text-generation-webui

• GPTQ-for-LLaMa

  19 129 7.7 Python

  4 bits quantization of LLaMa using GPTQ (by oobabooga)

git clone https://github.com/oobabooga/GPTQ-for-LLaMa

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• mlc-llm

  89 16,955 9.9 Python

  Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.

  

Have You tried https://mlc.ai/mlc-llm/ ? It uses Vulkan instead of CUDA, so running it is much easier, but only models compiled for MLC will work with it.

• koboldcpp

  180 3,749 10.0 C++

  A simple one-file way to run various GGML and GGUF models with KoboldAI's UI

Suggest a related project