FlexGen: Running large language models on a single GPU

• text-generation-webui

  876 36,552 9.9 Python

  A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

OK, looks like the "LLM retard guide" is "run this installer": https://github.com/oobabooga/text-generation-webui/releases/...

That's the only thing that worked for me, but it was so easy and worked instantly.

• FlexGen

  39 9,007 3.0 Python

  Running large language models on a single GPU for throughput-oriented scenarios.

  

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• llama.cpp

  773 57,463 10.0 C++

  LLM inference in C/C++

Copy paste the parts of the install steps for LLaMA C++ into ChatGPT and ask it to explain things simply and include any prerequisite steps you might need to do. If you get stuck, just ask ChatGPT and include any error messages.

https://github.com/ggerganov/llama.cpp

Suggest a related project